Release Notes - Salesforce Adapter v1.1

Owned by Arielle Kaye Odeste
Nov 21, 2023
2 min read

Overview

SalesForce 1.1 is a new version of the popular SalesForce integration that expands the flexibility of emite for custom reporting that includes SalesFroce data which needs to be joined in customer-specific use-cases. As ElasticSearch is optimized for speed, it is not always possible to reach the levels of flexibility our customers need where SQL is a better solution and still provides good performance for historical reporting. This functionality enables our consulting team to write custom SQL query that joins or augments the data to meet our customer's more complex needs and then pull this back into emite as a new fact query for reporting needs and further customizations.

It introduces the new onboarding framework where adapter can do incremental data onboarding to SQL. This version also allows the usage of SqlBulkCopy feature where records can be onboarded in batch reducing the processing time. It uses fewer database resources than normal SQL insert, making it more efficient and less likely to overload the database.

Another change is that adapter should poll from the last savepoint. It is dependent on where the data has been onboarded.

Product Enhancements

  • Implement SQL Data Onboarding Functionality and Create Data Onboarding Library - Adapter will be able to do incremental data onboarding to SQL.

  • Support Multiple Storage Types - SalesForce will be able to onboard data to multiple destination.

Adapter configs:

  • DataRepository: SqlServer|ElasticSearch

  • StorageType: ElasticSearch

This setting prevents duplicate data onboarding from both the old and new adapter framework. If StorageType is set to SqlServer, it will cause an infinite loop during polling of data because the old adapter framework does not update.

Bug Fixes

  • Implement the Last Update Functionality in the Adapter - This enables the adaptor to know the last date onboarded and continue polling for there, resolving the issue of repolling the same data repeatedly.

  • Savepoint update is not working as expected for SalesForce adapter - Adapter polls from the last savepoint. As this is dependent on where the data is onboarded to, please note the following:

  • Onboarding data to ElasticSearch only OR to both SqlServer and ElasticSearch, the last savepoint is found in the FQ lastdate field.

  • Onboarding to SqlServer only, the last save point is found in the LastDateTimeFieldValue field in the LastSavePoint table.

  • Adapter config for continuous polling:

    • PollingAllData = false

    • FixedPollStartTimeAsMinDate = false

    • MinDate = <>

  • The Id and SystemModstamp are required fields to be added in the SOQL query for the last savepoint and the primary index field

  • ESD-3981 Index emite_sf_opportunity stopped polling data - Change made to the core platform is to use the last date from the date controller field and not from any other field date. Last date must be the maximum value of the field that is defined as the date controller. Start time will be correct and no future dates in the logs. Moreover, polling will not stop.

  • ESD-4765 Salesforce adapter error - Time out error (A task was canceled) will not happen. Add AliasFieldNames if it already exists in the SOQL query.

  • ESD-5043 Recurring Salesforce adapter - Time out error (A task was canceled) will not happen.