Hi,

my production outsystems Version 10.0.823.0  on premise server often encountered this ALARM in the outsystems scheduler service. When this occurs, the email process becomes pending. I would need to ask the server administrator to reboot and the issue resolve temporarily

Can help how to resolve this?

OutSystems Scheduler Service Status: Warning: LightEvents Fetcher () is in status Processing for 32 seconds which exceeds the 27 seconds warning threshold (90% of the 30 seconds timer timeout definition). 

OutSystems Log Service Status: Warning: Bulk Extension Logger is in status Flushing messages for 17 seconds which exceeds the 10 seconds warning threshold. 


In the windows server event log, it shows the following error 

Log Name:      Application

Source:        OutSystems Deployment Controller Service

Date:          7/16/2020 4:00:15 PM

Event ID:      0

Task Category: None

Level:         Warning

Keywords:      Classic

User:          N/A

Computer:      169

Description:

Error opening connection to the database: Connection request timed out

Retrying...


   at OracleInternal.ConnectionPool.PoolManager`3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)

   at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)

   at OracleInternal.ConnectionPool.OracleConnectionDispenser`3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword)

   at Oracle.ManagedDataAccess.Client.OracleConnection.Open()

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.BaseTransactionService.GetConnection()

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.BaseTransactionService.CreateConnection(Int32 numRetries)

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.BaseTransactionService.CreateConnection(Int32 numRetries)

   at OutSystems.HubEdition.DatabaseProvider.Oracle.TransactionService.TransactionService.CreateConnection(Int32 numRetries)

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.GenericTransactionManager.BuildTransactionInfo()

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.GenericTransactionManager.AddToPoolAndReserve()

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.GenericTransactionManager.GetCommitableTransaction()

   at OutSystems.Internal.Db.DatabaseAccessProvider`1.GetCommitableTransaction()

   at OutSystems.HubEdition.RuntimePlatform.SoftwareUnitsReport.IsStoredDataValid(Int32 eSpaceId, Int32 eSpaceVersionId, String hmacKey)

   at OutSystems.Activation.Logic.Validations.ValidationSoftwareUnits.ShallRefreshObjectFeatureUsage(ObjectKey key, IDataReader reader)

   at OutSystems.Activation.Logic.Validations.AbstractHashedValidation.RecalculateStateInner(Transaction trans, Boolean ignoreCache)

   at OutSystems.Activation.Logic.Validations.ValidationSoftwareUnits.RecalculateStateInner(Transaction trans, Boolean refreshCache)

   at OutSystems.Activation.Logic.Validations.AbstractIntegerValuedValidation.RecalculateStateInner(Boolean ignoreCache)

   at OutSystems.Activation.Logic.Validations.Validation.RecalculateState(Boolean ignoreCache)

   at OutSystems.Activation.Logic.Validations.Validation.RecalculateState()

   at OutSystems.Activation.Logic.Server.RecalculateFeatureState(IValidation validation)

   at OutSystems.Activation.Logic.Server.RecalculateFeatureState(String featureId)

   at OutSystems.Activation.Logic.RefreshWorker.#t7b(String featureIdToRefresh)

   at OutSystems.Activation.Logic.RefreshWorker.#9r.#eac()

   at OutSystems.HubEdition.ServerCommon.SandboxUtils.ExecuteForSingleSandbox(String sandboxToExecute, Action action)

   at OutSystems.Activation.Logic.RefreshWorker.#r7b(String featureId, AutoResetEvent autoResetEvent)

   at OutSystems.Activation.Logic.RefreshWorker.#Br.#dac()

   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

   at System.Threading.ThreadHelper.ThreadStart()


Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

  <System>

    <Provider Name="OutSystems Deployment Controller Service" />

    <EventID Qualifiers="0">0</EventID>

    <Level>3</Level>

    <Task>0</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2020-07-16T08:00:15.000000000Z" />

    <EventRecordID>5646741</EventRecordID>

    <Channel>Application</Channel>

    <Computer>169</Computer>

    <Security />

  </System>

  <EventData>

    <Data>Error opening connection to the database: Connection request timed out

Retrying...


   at OracleInternal.ConnectionPool.PoolManager`3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)

   at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)

   at OracleInternal.ConnectionPool.OracleConnectionDispenser`3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword)

   at Oracle.ManagedDataAccess.Client.OracleConnection.Open()

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.BaseTransactionService.GetConnection()

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.BaseTransactionService.CreateConnection(Int32 numRetries)

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.BaseTransactionService.CreateConnection(Int32 numRetries)

   at OutSystems.HubEdition.DatabaseProvider.Oracle.TransactionService.TransactionService.CreateConnection(Int32 numRetries)

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.GenericTransactionManager.BuildTransactionInfo()

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.GenericTransactionManager.AddToPoolAndReserve()

   at OutSystems.HubEdition.Extensibility.Data.TransactionService.GenericTransactionManager.GetCommitableTransaction()

   at OutSystems.Internal.Db.DatabaseAccessProvider`1.GetCommitableTransaction()

   at OutSystems.HubEdition.RuntimePlatform.SoftwareUnitsReport.IsStoredDataValid(Int32 eSpaceId, Int32 eSpaceVersionId, String hmacKey)

   at OutSystems.Activation.Logic.Validations.ValidationSoftwareUnits.ShallRefreshObjectFeatureUsage(ObjectKey key, IDataReader reader)

   at OutSystems.Activation.Logic.Validations.AbstractHashedValidation.RecalculateStateInner(Transaction trans, Boolean ignoreCache)

   at OutSystems.Activation.Logic.Validations.ValidationSoftwareUnits.RecalculateStateInner(Transaction trans, Boolean refreshCache)

   at OutSystems.Activation.Logic.Validations.AbstractIntegerValuedValidation.RecalculateStateInner(Boolean ignoreCache)

   at OutSystems.Activation.Logic.Validations.Validation.RecalculateState(Boolean ignoreCache)

   at OutSystems.Activation.Logic.Validations.Validation.RecalculateState()

   at OutSystems.Activation.Logic.Server.RecalculateFeatureState(IValidation validation)

   at OutSystems.Activation.Logic.Server.RecalculateFeatureState(String featureId)

   at OutSystems.Activation.Logic.RefreshWorker.#t7b(String featureIdToRefresh)

   at OutSystems.Activation.Logic.RefreshWorker.#9r.#eac()

   at OutSystems.HubEdition.ServerCommon.SandboxUtils.ExecuteForSingleSandbox(String sandboxToExecute, Action action)

   at OutSystems.Activation.Logic.RefreshWorker.#r7b(String featureId, AutoResetEvent autoResetEvent)

   at OutSystems.Activation.Logic.RefreshWorker.#Br.#dac()

   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)

   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

   at System.Threading.ThreadHelper.ThreadStart()

</Data>

  </EventData>

</Event>


Log Name:      Application

Source:        OutSystems Platform

Date:          7/16/2020 4:00:20 PM

Event ID:      0

Task Category: None

Level:         Warning

Keywords:      Classic

User:          N/A

Computer:      169

Description:

Front-end Server 169 OutSystems Log Service Status: Warning: Bulk Request Logger is in status Flushing messages for 14 seconds which exceeds the 10 seconds warning threshold.


Message generated by eSpace ServiceCenter.

Event Xml:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">

  <System>

    <Provider Name="OutSystems Platform" />

    <EventID Qualifiers="0">0</EventID>

    <Level>3</Level>

    <Task>0</Task>

    <Keywords>0x80000000000000</Keywords>

    <TimeCreated SystemTime="2020-07-16T08:00:20.000000000Z" />

    <EventRecordID>5646744</EventRecordID>

    <Channel>Application</Channel>

    <Computer>169</Computer>

    <Security />

  </System>

  <EventData>

    <Data>Front-end Server 169 OutSystems Log Service Status: Warning: Bulk Request Logger is in status Flushing messages for 14 seconds which exceeds the 10 seconds warning threshold.


Message generated by eSpace ServiceCenter.</Data>

  </EventData>

</Event>

Hi Ronny,

I would suggest to open a new support case to let OutSystems help you with this problem.

https://www.outsystems.com/SupportPortal/CaseOpen/

Regards,

Daniel