We need a python script that extracts all tables and metadata from our Salesforce CRM organisation, and writes this data to zipped CSV files on Amazon S3.
1. Download all data from all Salesforce objects into a CSV file. The selection of files should be filtered using a Regex (positive / negative - to download only a subset of objects)
2. Download all binary data (salesforce attachements)
3. Download all metadata from Salesforce into a CSV file, with the following columns (“meta data column”, “data type”, “possible values”, “description”)
4. Upload the CSV files for data and meta data and the attachments to a directory on S3 bucket
- AWS SDK for Python is mandatory.
- All files on S3 should be zipped using GZip.
- Consistency of downloads is important / preferred over parallelization
- CSV files should include quotation / escaping of data
- Configuration should be done using configfilter (Salesforce OAuth, Bucket name, S3 folder, Regex for object filter) in separate ini file.
- Regex for filtering objects will be used to download only files from a specific namespace OR everything except files from a specified namespace (we need to separate HR relevant data which is stored in 2 different SFDC apps - Releco and TargetRecruit)
In addition to the previous requirements we need the following features:
- Error handling: log errors into a [leads].error.gz and standard (non-error) messages to [leads].log.gz
- Meta-data: we are not only interested in the data from the Salesforce objects (such as leads) but also on all data from other Salesforce REST interfaces (such as metadata of salesforce objects and attributes, users, and the like)
We do not provide an environment with data. We expect that the developer has access to a Salesforce environment (free developer edition is available) and S3 for testing purposes.
The configuration of the output CSV (delimiter, quote character) should be done via the ini file.
The Salesforce source changes frequently. The goal of this project is to develop a script that downloads all available fields and other data from the source present at execution time. a fixed set of attributes (to be configured in the ini file) is NOT SUFFICIENT to complete the project.
1) scan the rest api (or maybe the python SDK for SFDC) for all available attributes
2) write everything off to the CSV file
19 freelancers are bidding on average €153 for this job
IM a very patient person, trustworthy, responsible, reliable and you can count on me . im a data encoder for almost1 year in a company. i can make it in 2 days and i promise you that.