Asynchronous programming in .net Extensions

Asynchronous programming in .net Extensions

  

Does anyone have experience with or examples of using Asynchronous programming in OutSystems .net extensions?

I want to call the SendGrid (email) service. The SendGrid interface is request/response based. Here is their example C# code:

// using SendGrid's C# Library
// https://github.com/sendgrid/sendgrid-csharp
using SendGrid;
using SendGrid.Helpers.Mail;
using System;
using System.Threading.Tasks;

namespace Example
{
    internal class Example
    {
        private static void Main()
        {
            Execute().Wait();
        }

        static async Task Execute()
        {
            var apiKey = Environment.GetEnvironmentVariable("NAME_OF_THE_ENVIRONMENT_VARIABLE_FOR_YOUR_SENDGRID_KEY");
            var client = new SendGridClient(apiKey);
            var from = new EmailAddress("test@example.com", "Example User");
            var subject = "Sending with SendGrid is Fun";
            var to = new EmailAddress("test@example.com", "Example User");
            var plainTextContent = "and easy to do anywhere, even with C#";
            var htmlContent = "<strong>and easy to do anywhere, even with C#</strong>";
            var msg = MailHelper.CreateSingleEmail(from, to, subject, plainTextContent, htmlContent);
            var response = await client.SendEmailAsync(msg);
        }
    }
}

Pretty simple code. I have a concern that introducing the System.Thread.Tasks dependency and using async and await C# features may not play well with the OutSystems extensions.

I'm looking for any documentation or examples that would help me best understand how to use, if possible, asynchronous programming in a OutSystems extension.


Thanks



Hi Timothy,

Extensions run in the context of the eSpace that references them, and are to be used as simple libraries for one-shot tasks. I don't think you can reliably use them as background services or the like.

Hello,

First up - why are you concerned about using threading and/or async-await in OS extensions? As Kilian said, extensions are one-shot, but so far I've been able to use every trick I've had in my C# pockets.

In that example,

private static void Main()

is synchronous and will not return the control to Outsystems side of things until Execute.Wait() call returns. I would at least try implementing this action as Outsystems extension, basically defining this as an endpoint in integration studio and calling that Execute.Wait() from there.

Execute() in your example returns Task, for Task.Wait() documentation in MSDN, see https://msdn.microsoft.com/en-us/library/dd235606(v=vs.110).aspx

If this is something you would like to call from UI, then it's of course a blocking call. If this is the case and you are building a Mobile app, I'd see what JS promises could offer to you:

https://success.outsystems.com/Documentation/10/Extensibility_and_Integration/JavaScript/Extend_Your_Mobile_App_Using_JavaScript/Defining_Asynchronous_JavaScript_Code

Note that the link above applies only to OS Mobile apps. I've not so far used (or tried to use) JS promises in web side of things.

---

For this specific library, I see they also offer WebHooks approach (https://sendgrid.com/docs/API_Reference/Webhooks/event.html), which in some cases might be the preferred way to integrate with Outsystems.

Mikko Nieminen wrote:

Hello,

First up - why are you concerned about using threading and/or async-await in OS extensions? As Kilian said, extensions are one-shot, but so far I've been able to use every trick I've had in my C# pockets.

In that example,

private static void Main()

is synchronous and will not return the control to Outsystems side of things until Execute.Wait() call returns. I would at least try implementing this action as Outsystems extension, basically defining this as an endpoint in integration studio and calling that Execute.Wait() from there.

Execute() in your example returns Task, for Task.Wait() documentation in MSDN, see https://msdn.microsoft.com/en-us/library/dd235606(v=vs.110).aspx


If this is something you would like to call from UI, then it's of course a blocking call. If this is the case and you are building a Mobile app, I'd see what JS promises could offer to you:

https://success.outsystems.com/Documentation/10/Extensibility_and_Integration/JavaScript/Extend_Your_Mobile_App_Using_JavaScript/Defining_Asynchronous_JavaScript_Code

Note that the link above applies only to OS Mobile apps. I've not so far used (or tried to use) JS promises in web side of things.

---

For this specific library, I see they also offer WebHooks approach (https://sendgrid.com/docs/API_Reference/Webhooks/event.html), which in some cases might be the preferred way to integrate with Outsystems.


Mikko - Clarifications: 

- This is a back-end call - nothing to do with mobile.

- This is just a one-shot interface. SendGrid brings in all the async-await scaffolding. I'm just concerned that introducing async processing might compromise the OutSystems extension. 

After my first post, I had refactored the code, pretty much as you suggested:

namespace OutSystems.NssSendGridExt {
public class CssSendGridExt: IssSendGridExt {
public void MssSendTestEmail(out string ssResult) {
            TaskSendTestEmail().Wait();
            ssResult = "";
// TODO: Write implementation for action
} // MssSendTestEmail
        static async Task  TaskSendTestEmail()
        {
            var apiKey = "---------redacted-----------";
            var client = new SendGridClient(apiKey);
            var from = new EmailAddress("tmay@inscico.com", "Example User tmay");
            var subject = "Test02: Sending with SendGrid is Fun";
            var to = new EmailAddress("tmay@inscico.com", "Example User tmay");
            var plainTextContent = "and easy to do anywhere, even with C#";
            var htmlContent = "<strong>and easy to do anywhere, even with C#</strong>";
            var msg = MailHelper.CreateSingleEmail(from, to, subject, plainTextContent, htmlContent);
            var response = await client.SendEmailAsync(msg);
        }
    } // CssSendGridExt
} // OutSystems.NssSendGridExt

The code does send an email but it never returns to the caller. I don't see anything strange in this simple code but I'm not familiar with the innards of C# async. My next step would be to write a little standalone scaffolding program to test all of this without the OutSystems overhead. I'm curious as to why this never returns and would want to make sure it is something in my code and not an unwelcome side-effect of using async in an Outsystems Extension.

Also, after reading your post I think I will try using the SendGrid REST API to send emails and WebHooks to get notifications of delivery status etc. That would avoid having to use an Extension. I'll let you guys know how that turns out.

Thanks!


Hi Timothy,

I wonder whether what you're doing is really needed in an Extension? Sending e-mail can be done from the platform, so why put it in an Extension?

Kilian Hekhuis wrote:

Hi Timothy,

I wonder whether what you're doing is really needed in an Extension? Sending e-mail can be done from the platform, so why put it in an Extension?

I have used the email capabilities of OutSystems and they are fine for small batches of emails. However, I was interested in using the full breadth of the SendGrid API, not just sending a single email. Read on if you are interested in a little background on email marketing and SMTP relay services like SendGrid.

My application sits between an email marketing firm and services like SendGrid (which provide awesome email sending, tracking and management capabilities) and needs to process very large numbers of emails.

So the ref architecture is: Email Marketing -> Our Application -> SendGrid -> Consumer

Often, email marketing firms don't send emails directly, instead they send lists of email addresses, a template, name/value pairs and some substitution rules to a firm like SendGrid that then construct and deliver the individual emails and subsequently track KPIs like opens, clicks, blocks, spam, opt-out, etc. 

Sometimes the email marketing firm will construct full emails and send them, already personalized, to firms like SendGrid to take advantage of the tracking features. Here, I could use the email capabilities of OutSystems to call the SendGrid SMTP interface (but volume might be an issue).

Also, firms like SendGrid monitor the "reputation" of the sender and help senders avoid being blacklisted (this is a really big deal). If you ever need to do large email blasts (I've built email marketing systems that call SendGrid to routinely process million recipient campaigns) then I can vouch for SendGrid as a super technology partner.

FWIW, I'm only doing a POC in OutSystems as the platform isn't really the right tool to handle large, fully constructed, email pass-through campaigns (100,000+ per). However, we are using OutSystems for much of the rest of the application suite so doing a quick POC using OutSystems makes sense. Eventually, we may write the pass-through service in C#, Java, GO or node.js and have that service transact against our database directly (we use AWS RDS databases for our OutSystems applications). We might also just partner with SendGrid and have them extend their own service to do our bidding ;).

I can't share what we accomplish by sitting in-between the Email Marketing firm and SendGrid because that is a secret sauce. 

Sorry for the long answer to your short question! 

BTW, as I mentioned in my response to Mikko, I'm testing using the SendGrid API RESTful service now as that probably makes much more sense. I'm just used to calling SendGrid from services written in Java so I gravitated to an extension approach too quickly just to use the SendGrid .NET library. Still, being curious, I may play with the extension using that .NET library to figure out why it is choking on the async. 

 


Thanks for the comprehensive answer. One thing though:

"the platform isn't really the right tool to handle large, fully constructed, email pass-through campaigns (100,000+ per)." - I'm not sure I agree. It depends on the use case of course, but there's no obvious reason the platform couldn't handle this volume per se.

OK, good news:

have done this and it can work just fine.

I wrote a heavily multi-threaded extension (firing dozens of threads off at once to parse out a document, run search engine queries in parallel, collate the results, download the text from the returned URLs, perform additional analylsis).

The way I did it, was to have an Action in my Extension that accepted all of the needed information. In it, I did all of the needed queries (this was before asyc/await, I was using the multi-threaded ForEach from TPL), returned to the main thread to collate results, then used the same pattern to download and analyze the results, then collated the analysis into a structure as output.

To call this (because it could take a while), I used a timer. The UI was effectively building a record to be batched, then kicking off the timer to pick up the batches and run them. The new lightweight processes in BPT would be perfect for this instead of the timer, by the way.

J.Ja

Timothy May wrote:

Mikko Nieminen wrote:

After my first post, I had refactored the code, pretty much as you suggested:

namespace OutSystems.NssSendGridExt {
public class CssSendGridExt: IssSendGridExt {
public void MssSendTestEmail(out string ssResult) {
            TaskSendTestEmail().Wait();
            ssResult = "";
// TODO: Write implementation for action
} // MssSendTestEmail
        static async Task  TaskSendTestEmail()
        {
            var apiKey = "---------redacted-----------";
            var client = new SendGridClient(apiKey);
            var from = new EmailAddress("tmay@inscico.com", "Example User tmay");
            var subject = "Test02: Sending with SendGrid is Fun";
            var to = new EmailAddress("tmay@inscico.com", "Example User tmay");
            var plainTextContent = "and easy to do anywhere, even with C#";
            var htmlContent = "<strong>and easy to do anywhere, even with C#</strong>";
            var msg = MailHelper.CreateSingleEmail(from, to, subject, plainTextContent, htmlContent);
            var response = await client.SendEmailAsync(msg);
        }
    } // CssSendGridExt
} // OutSystems.NssSendGridExt

The code does send an email but it never returns to the caller. I don't see anything strange in this simple code but I'm not familiar with the innards of C# async. My next step would be to write a little standalone scaffolding program to test all of this without the OutSystems overhead. I'm curious as to why this never returns and would want to make sure it is something in my code and not an unwelcome side-effect of using async in an Outsystems Extension.

Also, after reading your post I think I will try using the SendGrid REST API to send emails and WebHooks to get notifications of delivery status etc. That would avoid having to use an Extension. I'll let you guys know how that turns out.

Thanks!


Hi Timothy, 

I', having the same problem !

Something as simple as 

public void MssmyTask(out string ssResult) {
            myTask.Wait();
            ssResult = "done";
}
        static async Task  myTask()
        {
              Task.Run(() =>
              {
                  int i = 0;
                  while (i < 5)
                  {
                      Console.WriteLine(i);
                      i++;
                      Thread.Sleep(500);
                  }
              }
              );
        }


Won't ever return!
Did you manage to find a workaround ?

Filipe,

Sorry - I did not. I used REST calls instead. I see others in the thread have succeeded though. 

Timothy , i just made it! You have to use .ConfigureAwait(false);
"Await resumes execution in the Synchronization context (essentially the thread) that started awaiting. If that thread is already blocked, eg. by awaiting itself on something else, the code deadlocks.": https://stackoverflow.com/questions/17202709/iis-failing-when-await-task-used