Hi Vikas,
You can do it delete or inactive by your logic, but if you delete user then you will have to problem after login becasue many place and things handle by User entity.
But this in application User entity use many places for capture user details like user id,User name etc.
And session/site property also haldle by it.
Regards
Rahul