That's true.
The "best" way is then to get the users without that Role.
You can do that by importing the dependency for User_Effective_Role entity from System (view with the roles each user has):

Then your query should have the following WITH OR WITHOUT join between User and User_Effective_Role entity with the Manager role on the JOIN condition:

And now add the filter to User_Effective_Role.User_Id = NullIdentifier():

In this way, you are getting the users who don't have the role Manager.
Regards,
João