The Xplenty API provides functions for controlling and monitoring Xplenty clusters and jobs. After defining an Xplenty data processing package using the Xplenty web application, you can call the Xplenty API to:
- create clusters
- run jobs
- monitor their progress
- terminate jobs and clusters
You can choose to use the Xplenty REST API, or one of its wrapper Libraries.
These are the topics covered on this page:
- Getting Started
- Xplenty Terminology
- REST Interface Specifications
- Libraries
- Security
- Collection Resources and Pagination
- Rate Limits
- API Resources
- Terms of Service
- References
For a quick overview of how to get started with the Xplenty REST API, you can read:
## Xplenty TerminologyThese are some of the terms you will encounter in the Xplenty API documentation.
An Xplenty package is a data flow definition that you define in the Xplenty web application. It describes the data to process (location, schema, fields), data manipulation to perform, and the output destinations (location, schema). The package's workflow is implemented by jobs.
An Xplenty job is a process that is responsible for performing a data flow according to a specific package on a cluster. The job is a batch process that runs on a finite amount of data and then terminates. Several jobs can run the same package simultaneously. When you call the Xplenty API to run a new job, you supply the name of the package whose workflow the job should perform, and the cluster on which to run.
An Xplenty cluster is a group of machines (nodes) that is allocated exclusively to your account's users. You can create one or more clusters, and you can run one or more jobs on each cluster. A cluster that you've created remains allocated to your account until you request to terminate the cluster.
An Xplenty schedule executes packages periodically starting at a specified date and time. The packages will be executed as scheduled, using an existing cluster that fits the scheduled cluster size or if one doesn't exist, a cluster will be provisioned automatically with the number of specified nodes. By default, the cluster is taken down as soon as package execution is completed.
An Xplenty delivery is a blueprint for a set of resources which automates the task of delivering data from one place to another.
An Xplenty account represents a related group (usually a company) of Xplenty users. An account is created when the user signs up to use the Xplenty service. An API key is generated for the user, which must be supplied when calling the Xplenty API.
## REST Interface SpecificationsThe Xplenty REST API conforms to the design principles of Representational State Transfer (REST).
To get a quick start with the Xplenty API, we recommend using the curl command-line utility.
All URLs start with: https://api.xplenty.com/{account_id}/api/
Parameter values should be converted to UTF-8 and URL encoded.
All Xplenty APIs support jsonp, which is the json format with a callback specified, such as:
callback=callback_method
All date parameters will be treated as UTC and their formats must be in ISO 8601 format:
YYYY-MM-DDTHH:MM:SSZ
All published date and time objects are UTC based and returned in ISO 8601 format:
YYYY-MM-DDTHH:MM:SSZ
The API presently supports the JSON format only. Specify a custom mime type in the [Accept] header as follows:
application/vnd.xplenty+json
Unless you specify a version, the latest representation of resources will always be returned in the response. If you’re building an application and want to ensure a consistent response format, specify a version as follows:
application/vnd.xplenty+json; version=2
Many API methods take optional parameters. For GET requests, any parameters not specified as a segment in the path can be passed as an HTTP query string parameter:
$ curl -i "https://api.xplenty.com/xplenty-account/api/clusters?status=available"
In this example, the 'xplenty-account' value is provided for the :account_id parameter in the path while the 'available' value is provided for :status that is passed in the query string.For POST, PATCH, PUT, and DELETE requests, parameters not included in the URL should be JSON encoded:
$ curl -X POST -u api_key "https://api.xplenty.com/xplenty-account/api/clusters" \
-H "Accept: application/vnd.xplenty+json; version=2" \
-H "Content-Type: application/json" \
-d '{
"name": "my cluster",
"description": "production cluster",
"type": "production",
"nodes": 2
}'The Xplenty API supports Cross Origin Resource Sharing.
You can read the CORS W3C working draft, or this introduction from the HTML 5 Security Guide.
The latest version of the Xplenty API (V1) supports CORS requests from any domain.
HTTP Status Codes
These are the HTTP status codes that the Xplenty API can return:
- 200 OK: Request succeeded.
- 201 Created: The requested resource was created successfully.
- 204 No Content: Request succeeded. No content is returned.
- 304 Not Modified: There was no new data to return.
- 400 Bad Request: The request was invalid. An accompanying error message will explain why.
- 401 Unauthorized: You are attempting to access the API with invalid credentials.
- 402 Payment Required: You must confirm your billing info to use this API.
- 403 Forbidden: The request has been refused. An accompanying error message will explain why.
- 404 Not Found: The URI requested is invalid or the resource requested does not exist.
- 406 Not Acceptable: The requested mime-type is not acceptable.
- 415 Unsupported Media Type: The specified media type is not supported.
- 422 Unprocessable Entity: You have sent invalid fields.
- 429 Too Many Requests: The request exceeded the rate limitations.
- 500 Internal Server Error: An internal error occurred in the request.
- 502 Bad Gateway: Xplenty is down or being upgraded.
- 503 Service Unavailable: The Xplenty servers are up, but overloaded with requests. Try again later.
Error Responses
When the API returns an error message, it does so in your requested format. For example, an error from a JSON method might look like this:
{
"message": "Item not found."
}Use a wrapper in the official Xplenty library, or a third party library.
Java wrapper Python wrapper Ruby wrapper
### Third Party [.NET wrapper for Xplenty Rest API](https://github.com/dilievsky/xplenty.dll) ## SecurityXplenty provides all REST API methods over SSL. Whenever your code might be operating on a non-secure network (that is, if you're developing a client application), please make use of SSL for all authenticated or sensitive requests. For example, requesting cluster information should be performed by an Xplenty client over SSL. Service-to-service communication may not benefit from SSL if you trust your hosting provider (or if you are your own hosting provider).
Most of the Xplenty API calls require authentication, supplied in the form of the API key which is generated for each account.
In order to get your API key, view your user's personal information page. See here for more information.
Once you have an API key, you can use HTTP Basic Authentication with the API key as a username and a blank password. Here's an example using curl (the colon separates the username and password):
curl -H "Accept: application/vnd.xplenty+json" -u <apikeyhere>: https://api.xplenty.com/:account_id/clustersAPI key authentication works well for personal scripts, but is not recommended for third party services. We plan to deliver OAuth in the near future to provide better granularity and control when providing access to third party services.
## Collection Resources and PaginationThe response to a GET request for collection resources (e.g. clusters) may not return all the objects in the collection, depending on the number of objects. To query collection resources incrementally, use the following parameters:
- offset - the index of the first object to retrieve, starting from 0
- limit - the number of items to return (default is 20, maximum is 100)
Information about pagination is provided in the Link header of an API call, so that you know how many more requests are needed to retrieve the complete list of collection resources:
$ curl -I "https://api.xplenty.com/:account_id/api/clusters?offset=3&limit=10"The -I parameter indicates that we only care about the headers, not the actual content.
In examining the result, you'll notice some information in the Link header that looks like this:
Link: <https://api.xplenty.com/:account_id/api/clusters?offset=4&limit=10>; rel="next",
<https://api.xplenty.com/:account_id/api/clusters?offset=2&limit=10>; rel="prev"
rel="next" says that the next offset is offset=4 and rel="prev" says that the previous offset is offset=2. Using this information, you could traverse through the list of results in the API call.
The Xplenty API only allows clients to make a limited number of calls in a given hour. It uses a credit allocation system to ensure fair distribution of capacity. Each user is allocated 5,000 credits per hour. This is keyed off either your login or the request IP. If you exceed this limit, you'll get a "429 Too Many Requests" response for subsequent requests.
All Xplenty API methods are rate-limited except for querying the rate limit. You can check your rate limit status without incurring an API hit, as follows:
`GET /rate_limit_status`
Every time you call the API, the current rate status will be returned in the response under the following headers:
| HTTP Header | Description |
| X-RateLimit-Limit | Total credits allocated |
| X-RateLimit-Remaining | Total credits available |
Unless otherwise noted, an API call deducts 1 credit from your allocation.
When a client exceeds the allocated rate limit, the Xplenty API returns a "429 Too Many Requests" response with an associated "Rate Limit Exceeded" message as the error description.
Xplenty REST API whitelisting is not supported.
We ask that you honor the rate limit. If your application abuses the rate limit your user will be blacklisted, and you will be unable to get a response from the Xplenty API.
If your user has been blacklisted and you think there has been an error, you can contact the email address on the API support page. So we can get you back online quickly, please include the following information:
- If you are using the REST API, make a call to the /rate_limit_status from the computer which is blacklisted and include the response in your email.
- Explain why you think your application was blacklisted.
- Describe in detail how you have fixed the problem that you think caused you to be blacklisted.
These are the methods supported by the Xplenty API:
- Create Cluster
- List Clusters
- Search Clusters
- Update Cluster
- Get Cluster Information
- Terminate Cluster
- Run Job
- List Jobs
- List Job Children
- Search Jobs
- Get Job Information
- Terminate Job
- Watch Clusters and jobs
- Get Job Execution Variables
- Preview Job Output
- List Packages
- Search Packages
- Update Package
- Export Package
- Run Package Validation
- Get Package Validation Information
- List Package Validation
- Create Package
- Create Schedule
- Get Package Information
- Delete Package
- List Schedules
- Search Schedules
- Update Schedule
- Get Schedule Information
- Delete Schedule
- Watch Schedules
- Run Schedule
- Get Account
- Create Account
- List Accounts
- Update Account
- Delete Account
- Get Account Payment Method
- Add Account Member
- Get Account Member Information
- List Account Members
- Update Account Members Role
- Delete Account Member
- List Connection Types
- List Account Connections
- Create Account Connections
- Test New Account Connection
- Update Account Connections
- Get Account Connection
- Delete Account Connections
- Test Connection
- Validate Connection
- Import Connection schema
- List Bing Ads Accounts
- List Bing Ads Columns
- List Salesforce Objects
- List Salesforce Fields
- List Facebook Ads Insights Accounts
- List Facebook Ads Insights Fields
- List Google Analytics Accounts
- List Google AdWords Customers
- Get Authenticated User Information
- Regenerate Authenticated User Tokens
- Update Authenticated User
- Create Public Key
- List User Public Keys
- Delete User Public Key
- Get User Public Key Information
- List User Notifications
- Mark User Notifications as read
- Reset User Password
- Get Subscription Information
- List Supported Stacks
- List Available Regions
- List Available Account Regions
- List System Variables
- List Product Updates
- List Hook Types
- List Hooks
- Search Hooks
- Get Hook Information
- Create Hook
- Update Hook
- Ping Hook
- Reset Hook Salt
- Delete Hook
- List Supported Time Zones
- List Invoices
- Get Invoice Information
- Help
- List Deliveries
- Get Delivery Information
- Create Delivery
- Delete Delivery
Please refer to our Terms of Service page.