I missed that part.
You have to add a handler on the OnChange event of the Upload widget:

And there, you can get the extension from the name and do the validation you want. If the extension is not the right one, you reset the Name and BinaryDataVar variables and show an error message to the user:

I don't know which extensions you want, so my example and OML in attachment considers any extension other than .jpg and .png as invalid.