Hi everyone, 

We're having trouble in debugging web applications on our enterprise environment. We have our development environment setup with a load balancer with one deployment controller and one front-end server. 

Debugging doesn't work most of the time when service studio is connected with load balancer url.

I was going through forum posts about the debugging issue and found this. After trying the suggestion of using IP address I was able to debug the application by connecting to the second server(one ending with 120) whereas when connecting to first server(ending with 19) debugger won't even start.

I'm not sure what to make of it and how to use this information to fix the debugging issue permanently. Do I need to change the load balancer to use a single server? or enable sticky sessions(this still has a potential issue when the user gets connected to the server where debugger doesn't work).

Thanks in advance.


My understanding is that load balancing is only supported in Production mode, and you can't debug in production mode. At least that's what support keep telling us when we wanted to do specific testing with multiple front ends.

Actually just reread your post. If you only have one front end server then your not using load balancing and shouldn't have an issue. It's the front end server that your debugger is connecting to so if there is only one then it should be fine. If you have two servers listed but only one active then I would suggest contacting support and getting them to remove the unused server.


John Williams wrote:

Actually just reread your post. If you only have one front end server then your not using load balancing and shouldn't have an issue. It's the front end server that your debugger is connecting to so if there is only one then it should be fine. If you have two servers listed but only one active then I would suggest contacting support and getting them to remove the unused server.


Hi John,

Thanks for the reply. I just went in to the server and checked configuration tool. It appears the first server which appears to be unused is configured as dedicated Deployment Controller and doesn't host any application. So I can't really remove it from the setup or can I?


Ok thats unusual that load balancing would be directing debugger traffic to that then, I think the load balancer setup may be at fault. Is this a PaaS or self hosted platform? If it's the PaaS option I would be contacting support as it shouldn't be stopping your debugger from working.

John Williams wrote:

Ok thats unusual that load balancing would be directing debugger traffic to that then, I think the load balancer setup may be at fault. Is this a PaaS or self hosted platform? If it's the PaaS option I would be contacting support as it shouldn't be stopping your debugger from working.


We're on a self-hosted platform. And If I remember correctly support had recommended this setup. So, should I be logging a support ticket to get a better solution?

I think your issue may be that the load balancer setup itself isn't quite right. If there is only one front end server then it should be directing all debug trafic to that server. I don't know what ports the debugger uses but it may be as simple as finding the correct port and updating the load balancer. Support may be able to help with that. Are you using a .net based install? I think it might be 4022 looking at the below


https://docs.microsoft.com/en-us/visualstudio/debugger/remote-debugging-aspnet-on-a-remote-iis-7-5-computer?view=vs-2017

John Williams wrote:

Ok this one has a bit more info on the IIS debug ports and firewalls.

https://docs.microsoft.com/en-us/visualstudio/debugger/configure-the-windows-firewall-for-remote-debugging?view=vs-2017


I think it's more to do with the load balancer than the firewall and remote debugging because I'm able to debug when connecting directly to the front-end server. 

The strange thing is: we're using an application load balancer that has only one server in the target group and that's the one I'm able to debug with when connected directly. So is it the security group on the ALB or the port? It works occasionally so I'm not able to pinpoint the exact thing here.


Yes but the load balancer will need to be setup for the same ports. I just included that link as it had a list of ports that the debugger uses.

Hi Sunil,


There is an option on Service Studio preferences that switches the debugger to use one connection for request.

It's a bit slower, but it's more load balancer friendly.


@jonh Service Studio does not use VS debugging tools, that would require very high permissions on the remote machine. All communication is done in the normal http(s) ports.


Regards,

João Rosado

João Rosado wrote:

Hi Sunil,


There is an option on Service Studio preferences that switches the debugger to use one connection for request.

It's a bit slower, but it's more load balancer friendly.


@jonh Service Studio does not use VS debugging tools, that would require very high permissions on the remote machine. All communication is done in the normal http(s) ports.


Regards,

João Rosado

Hi Joao,

We tried the use one connection for request setting but that doesn't help. I have opened a support case with OutSystems to help us out. Will update once we have a fix for everyone to know.


Thanks,

Sunil



Hi Sunil,


There is a workaround for it:

  1. You access on web browser to one of the servers in specific (for instance: https://<Server1>/<Module>/<Screen>.aspx) instead of the load balancer;
  2. In Service Studio, you connect to that same server and your debug should work.


As I said this is a workaround and it worked for me when I had the same situation.


Cheers,

João

Solution

Here's the response from OutSystems support that solved the issue for us. Just to make sure that people facing the same issue can find the resolution. 

We have been analyzing this issue, and we would like to share our findings with you:


  • As my colleague ******** previously mentioned, a common cause for the reported misbehavior is that, during the debug session, the connection between Service Studio and the environment is closed by a network element (such as a firewall). Nevertheless, you already tried to configure the option Use One Connection per Request in Debugger, which did not solve the problem. Therefore, we are lead to disregard that hypothesis.

  • Given the description of the issue, we currently suspect that the problem is caused due to concurrent debugging sessions on the same eSpace. Please let us clarify: when a developer is debugging an eSpace, it sends a request (in your case to the Load Balancer) which in turns forwards it to the server; then, the server replies back to the Load Balancer, but the response from the server may not be aware of the machine that created the original request. Thus, the Platform cannot distinguish between each user, and consequently, the user that made the request may not be the one to which the breakpoint is reported to.

    Moreover, the fact that the debugger works properly when you connect directly to the IP front-end, seems to reinforce the above suspicion.

  • Fortunately, we have a configuration that you can apply to resolve such scenarios: using Trusted proxies.
    You can configure a list of trusted proxies IP addresses or IP address ranges to let the OutSystems platform know that it can trust values passed in the X-Forwarded-For HTTP header. That way, the platform will know the origin of the requests.

    In order for you to configure trusted proxy in your on-premises environment, please proceed as follows:

    1. Access Service Center in your environment;
    2. Go to Administration > Security > Network Security;
    3. Add the IP addresses or IP ranges for trusted proxies in the Trusted proxy addresses field
  • Keep in mind that the IP addresses that you configure here should be the addresses assigned to the proxy’s network interfaces that communicate in the same network where the OutSystems servers are. A common scenario in the presence of a Load Balancer is for that LB to have network interfaces for the public network and different network interfaces for the internal network, which is usually where OutSystems servers are.

    A fail-proof way to know if the configured IP addresses are the correct ones is to remotely access one of the OutSystems front-ends and get the IP address of the Proxy / Load Balancer from there, for example using the nslookup command.

    The command will return the list of IP addresses available for that proxy, which are the ones that the front-end can communicate with.

Solution