Skip to content

Commit

Permalink
Wait by default. Reutrn exit code based on upload status. Fixes #3
Browse files Browse the repository at this point in the history
  • Loading branch information
nithinphilips committed Dec 17, 2014
1 parent 00c9f03 commit 6f24ebc
Show file tree
Hide file tree
Showing 2 changed files with 27 additions and 16 deletions.
14 changes: 6 additions & 8 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -118,18 +118,16 @@ enter ``tririga.example.com:80``, Odel resolves this to

Waiting for Processing
----------------------
Normally Odel will terminate as soon as the file is transmitted to Tririga. It
will take Tririga a few minutes to process the file and create the records.
Often, when running as part of a batch process you will want to wait until the
file is processed before performing the next task.

If the ``-w`` flag is set, Odel will wait until Tririga changes the data upload
status to *Rollup All Completed* or *Failed*, indicating the completion of the
upload process.
By default Odel will wait until Tririga changes the data upload status to
*Rollup All Completed* or *Failed*, indicating the completion of the upload
process.

This only waits for creation of records. Tririga may still continue to process
*Associate* and other asynchronous tasks in the background.

To disable this a quit as soon as the upload is complete, specify the
``-no-wait``` flag.

Building Windows Installer
--------------------------
Windows installer can be built on Windows machines. You will need Python 2.7 (Windows version)
Expand Down
29 changes: 21 additions & 8 deletions odel/diuploader.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,10 +239,9 @@ def parse_url(url, port='9080'):


@arg(
'--wait', '-w',
help="Wait and keep running until the file is processed by Tririga. Odel "
"will poll Tririga to check the record status. Useful for batch "
"processing"
'--no-wait', '-w',
help="Do not Wait until the file is processed by Tririga. By default Odel "
"will wait until Tririga has fully processed the file.",
)
@arg('--module', '-m', help="The name of the module to which to upload")
@arg('--businessobject', '-b',
Expand All @@ -262,7 +261,7 @@ def parse_url(url, port='9080'):
"appended")
def upload(url, filename, username="system", password="admin",
module=None, businessobject=None, form=None, action=None,
wait=False):
no_wait=False):
"""
Uploads a file to Tririga Data Integrator.
Expand Down Expand Up @@ -361,9 +360,13 @@ def upload(url, filename, username="system", password="admin",
url = '{}/html/en/default/common/dataSmartUpload.jsp'.format(site_url)
response = session.post(url, params=diparams)

if wait:
ecode = 0

if not no_wait:
logging.debug("Waiting for the processing to complete.")
wait_for_upload(filenameonly, site_url, username, password)
ecode = wait_for_upload(filenameonly, site_url, username, password)

sys.exit(ecode)

class MultipartMimeFilter(MessagePlugin):
"""
Expand Down Expand Up @@ -483,12 +486,20 @@ def wait_for_upload(filename, site_url, username, password):
start = 1
maximumresultcount = 999

ok_status = ("Rollup All Completed", "Failed")
status_codes = {
"Rollup All Completed": 0,
"Failed": 10,
"NEW": 20,
"DONE": 30,
"UPLOADING...": 40
}

processing_status = ("NEW", "DONE", "UPLOADING...")

total_sleep_time = 0
retry = True
retries = 0
last_status = None

while retry and retries < MAX_RETRIES:
sleep_time = 2**retries / 100.0
Expand Down Expand Up @@ -519,6 +530,7 @@ def wait_for_upload(filename, site_url, username, password):
logging.debug("Uploaded record status: {}".format(column.value))
if column.value in processing_status:
found_processing = True
last_status = column.value

This comment has been minimized.

Copy link
@nithinphilips

nithinphilips Dec 17, 2014

Author Owner

May not be correct if there are multiple files with the same name. See #3


if not found_processing:
logging.debug("File appears to be fully processed")
Expand All @@ -528,4 +540,5 @@ def wait_for_upload(filename, site_url, username, password):

logging.debug("File processed in about {} seconds".format(total_sleep_time))

return status_codes[last_status]

0 comments on commit 6f24ebc

Please sign in to comment.