Salesforce Configuration File

Configuring the Salesforce Data Loader

For the Salesforce Reader and Salesforce Writer, there are some common configuration parameters which are used for scripting the Salesforce Data Loader.

These settings are stored in a separate XML file in the following format:

<?xml version="1.0" encoding="utf-8"?>
<SfdcConfig>
  <DataLoaderDir>[path to Data Loader]</DataLoaderDir>
  <LogFileDir>[log file path]</LogFileDir>
  <SuccessFileName>[full path to success log]</SuccessFileName>
  <ErrorFileName>[full path to error log]</ErrorFileName>
  <SfdcUsername>[Salesforce user name]</SfdcUsername>
  <SfdcPassword>[Salesforce Password + Token]</SfdcPassword>
  <SfdcEncryptedPassword>[encrypted password</SfdcEncryptedPassword>
  <SfdcEndPoint>[Salesforce endpoint]</SfdcEndPoint>
  <LoadBatchSize>[batch size]</LoadBatchSize>
  <KeepTempFiles>[true|false]</KeepTempFiles>
  <FailOnErrors>[true|false]</FailOnErrors>
  <FailOnFirstError>[true|false]</FailOnFirstError>
  <Timezone>[Timezone]</Timezone>
  <UseBulkApi>[true|false]</UseBulkApi>
  <BulkApiSerialMode>[true|false]</BulkApiSerialMode>
  <BulkApiZipContent>[true|false]</BulkApiZipContent>
</SfdcConfig>

Setting	Description
DataLoaderDir	DataLoader only: The path to the Salesforce Data Loader, e.g. C:\Program Files (x86)\salesforce.com\Data Loader
LogFileDir	DataLoader only: The path under which the Data Loader stores its logs.
SuccessFileName	The name of the CSV log file into which successful write operations are stored (such as insert, upsert, update or `delete).
ErrorFileName	The name of the CSV log file into which errored write operations are stored.
SfdcUsername	The user name of the user to use for interacting with Salesforce. This user needs to have API access enabled.
SfdcPassword	.Net only: The non-encrypted password, including the security token, for accessing Salesforce. This is what you normally would enter directly enter into the Data Loader. This tag only applies to the .Net Salesforce Plugin (Salesforce.Net Reader and Salesforce.Net Writer).
SfdcEncryptedPassword	The encrypted password, including the security token, for accessing Salesforce. See below. Applies to DataLoader plugins (Salesforce Reader and Salesforce Writer, not to the .Net plugins.
SfdcEndPoint	The Salesforce endpoint to talk to. Usually https://login.salesforce.com for production Orgs, or https://test.salesforce.com for sandboxes.
LoadBatchSize	The batch size of Salesforce write operations. Defaults to 200. The total request size must stay below 50 MB, so lowering this may be indicated at times.
KeepTempFiles	DataLoader only: Debugging setting for keeping the Data Loader configuration files after the fact.
FailOnErrors	Defaults to true. The error log of the Salesforce Write operation is checked for entries; if there are errored write operations, NFT will exit with an error code. If set to false, errored operations will be ignored.
FailOnFirstError	.Net Only: Set this to true in order to make NFT abort the processing after the very first error has been detected (disregarding the operation). This only applies to the .Net plugin (Salesforce.Net Writer). Please note that there may still be multiple errors being output, as the entire batch is always processed.
Timezone	The timezone date and time values are passed on into.
UseBulkApi	DataLoader only: Whether or not to use the SFDC Bulk API or not.
BulkApiSerialMode	DataLoader only: Whether or not to use the Bulk API serial mode (process batches one after the other). This may be useful if you encounter race conditions during the parallel processing of the records you did not experience without the Bulk API mode
BulkApiZipContent	DataLoader only: Set to true when trying to upload Attachments using the Bulk API (and tell me if it works).

Example:

<?xml version="1.0" encoding="utf-8"?>
<SfdcConfig>
  <DataLoaderDir>C:\Program Files (x86)\salesforce.com\Data Loader</DataLoaderDir>
  <LogFileDir>..\..\output\§env§\§entity§</LogFileDir>
  <SuccessFileName>..\..\output\§env§\§entity§\§entity§_success_§operation§_log.csv</SuccessFileName>
  <ErrorFileName>..\..\output\§env§\§entity§\§entity§_error_§operation§_log.csv</ErrorFileName>
  <SfdcUsername>my.user.name@domain.com</SfdcUsername>
  <SfdcEncryptedPassword>ab3aa7674884bcf370115fe9b702297192e3777e1ec69b89d6a4a02356738e0af702a50afb7a44f1f80a8187cf4eec85</SfdcEncryptedPassword>
  <SfdcEndPoint>https://test.salesforce.com</SfdcEndPoint>
  <KeepTempFiles>false</KeepTempFiles>
  <FailOnErrors>true</FailOnErrors>
</SfdcConfig>

Obviously, the user name and passwords are fake here.

Using Command Line Parameters

In many cases, it is useful to use command line parameters inside the above settings, e.g. for creating different log files for different entities.

Example:

<?xml version="1.0" encoding="utf-8"?>
<SfdcConfig>
  ...
  <SuccessFileName>..\output\§entity§\§entity§_§operation§_success.csv</SuccessFileName>
  <ErrorFileName>..\output\§entity§\§entity§_§operation§_error.csv</ErrorFileName>
  ...
</SfdcConfig>

In this example, the SFDC Configuration file is parametrized using two parameters: entity and operation to get separate success and error log CSV files from the Data Loader for different kinds of entities.

NFT has subsequently to be called using these parameters to be able to resolve the file names (and will fail without the definition of these parameters).

C:\Temp> NoFrillsTransformation.exe load_accounts.xml entity=Account operation=upsert
C:\Temp> NoFrillsTransformation.exe load_accounts.xml entity=Contact operation=upsert

This will result in four different files being created:

• Account_upsert_success.csv
• Account_upsert_error.csv
• Contact_upsert_success.csv
• Contact_upsert_error.csv

Instead of constantly overwriting the success and error logs (which can be fatal).

### Salesforce Data Loader User Prerequisites

Note: This is only necessary for the DataLoader plugin; it does not apply to the .Net plugin, there you need to supply the non-encrypted password + token as the SfdcPassword tag.

The user profile used for interacting with Salesforce via the Data Loader must fulfill the following requirements:

• The user has to be API enabled (user setting)
• The user has to have a valid security token for use with the API (see Salesforce documentation)

The API access password can then be generated by concatenating the usual Salesforce password with the security token for a quite long string of characters.

In order to enable the command line interface (and thus NFT) for use with the Data Loader, the encrypt.bat command line tool (in the Data Loader's bin directory) has to be used:

C:\Program Files (x86)\salesforce.com\Data Loader\bin>encrypt -e KHLJSKFjiSFgKUD
zefIUEZRiuezoia47598we47598weztokesg
2015-07-10 12:02:24,298 INFO  [main] security.EncryptionUtil main (EncryptionUti
l.java:365) - 1490d3b63d54697e1d62d54b2b2c7bb0e4865961406a244e12f9363fa8366a909a
4cb4ca77458edc3b4ee4c126cb7262b65d7df7a47a4629

The resulting hex string is the string to insert into the above SfdcEncryptedPassword field.

Overriding Salesforce Configs

For some Salesforce setting, the Plugin reacts to specific command line parameters of NFT (see also [[Config File Documentation|Config File Documentation#parameters). By setting these parameters, some configuration settings can be overridden:

• sfdcloadbatchsize=<batch size>: Use this parameter to override the LoadBatchSize for specific entities. Useful e.g. for lowering the batch size for Attachments (which tend to be bigger and thus may break the 50 MB boundary per batch). In combination with usebulkapi=true, it may be useful to set this to something bigger than 200; for the Bulk API mode, the maximum value is 10000. Please note that the 50 MB size limit also applies to the Bulk API mode.
• sfdcfailonerrors=<true|false>: Use this parameter to override the default setting of failing on write errors.
• sfdckeeptempfiles=<true|false>: Useful when debugging a specific transformation which fails on the Data Loader bits.
• usebulkapi=<true|false>: Use this parameter if you want to use the Bulk API for a specific command line (and not generally)
• bulkapiserialmode=<true|false>: Use this parameter (in combination with usebulkapi=true) to switch on the Bulk API Serial mode for a specific command line
• bulkapizipcontent=<true|false>: Use this parameter (in combination with usebulkapi=true) to turn on the Bulk API ZIP content mode for a specific command line

Example:

C:\Temp> NoFrillsTransformation.exe sfdc_load.xml sfdcloadbatchsize=10
C:\Temp> NoFrillsTransformation.exe sfdc_load2.xml usebulkapi=true bulkapiserialmode=true sfdcloadbatchsize=5000