Skip to content

Commit

Permalink
Fix synapse errors not print out issue (feathr-ai#734)
Browse files Browse the repository at this point in the history
Co-authored-by: enya-yx <enya@v-ellinlu-2.fareast.corp.microsoft.com>
  • Loading branch information
2 people authored and hyingyang-linkedin committed Oct 25, 2022
1 parent 529da26 commit 9d5b98d
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,7 @@ def create_spark_batch_job(self, job_name, main_file, class_name=None,
def get_driver_log(self, job_id) -> str:
# @see: https://docs.microsoft.com/en-us/azure/synapse-analytics/spark/connect-monitor-azure-synapse-spark-application-level-metrics
app_id = self.get_spark_batch_job(job_id).app_id
url = "%s/sparkhistory/api/v1/sparkpools/%s/livyid/%s/applications/%s/driverlog/stdout/?isDownload=true" % (self._synapse_dev_url, self._spark_pool_name, job_id, app_id)
url = "%s/sparkhistory/api/v1/sparkpools/%s/livyid/%s/applications/%s/driverlog/stderr/?isDownload=true" % (self._synapse_dev_url, self._spark_pool_name, job_id, app_id)
token = self._credential.get_token("https://dev.azuresynapse.net/.default").token
req = urllib.request.Request(url=url, headers={"authorization": "Bearer %s" % token})
resp = urllib.request.urlopen(req)
Expand Down

0 comments on commit 9d5b98d

Please sign in to comment.