download-db-log-file-portion with starting token of 0 not working as documented #3079
Comments
|
Interesting. Thanks for the debug logs that is really helpful. So it looks like the value that the service is returning is literally returning |
|
@kyleknap I wonder if the API has some changes where you can only request a response of a certain size. If that's the case the documentation for the aws cli should change to reflect that. If that's the case the current documentation is wrong. |
|
Looking at the API, you may have some luck with the |
|
Nope, setting the page size to a high value results in the exact same file output. The documentation you linked specifies It looks like if a user has a 500mb log file are are using the api they must make 500 aws cli calls. |
|
I also just assumed I could update the |
|
I had the same issue with both the CLI and the SDK. I ended up making a CLI program, rds-logs-download-url, to give me the URL for the full log file so I could just download it. I hope someone else find it useful as well until this issue gets revolved. |
diehlaws
added
guidance
|
Following up here, I don't think there's anything actionable on the CLI side. Kyle provided an explanation about what's happening, and also recommended to reach out to the forums. Closing issue due to inactivity. |
|
I've faced with the same issue. |
|
The problem still occurs. |
|
@jakubkalicki, I posted this up above, but as a workaround I wrote a cli in golang to programmatically get the download url if you are in need of something right away. rds-logs-download-url. You just need the RDS instance identifier and the name of the log to generate the signed URL. (I used the other aws commands to get these in a bash script and programmatically download the files to feed into something for analytics). I also hope they add this functionality into the CLI. |
|
If anybody is still running into this problem, I have a small Python script that you can use to download RDS log files and save them in S3: https://github.com/ccampo133/rds-logs-to-s3 |
* sam pipeline bootstrap (aws#2811) * two-stages-pipeline plugin * typos * add docstring * make mypy happy * removing swap file * delete the two_stages_pipeline plugin as the pipeline-bootstrap command took over its responsibility * remove 'get_template_function_runtimes' function as the decision is made to not process the SAM template during pipeline init which was the only place we use the function * sam pipeline bootstrap command * move the pipelineconfig.toml file to .aws-sam * UX - rewriting Co-authored-by: Chris Rehn <crehn@outlook.com> * UX improvements * make black happy * apply review comments * UX - rewriting Co-authored-by: Chris Rehn <crehn@outlook.com> * refactor * Apply review comments * use python way of array elements assignments * Update samcli/lib/pipeline/bootstrap/stage.py Co-authored-by: _sam <3804518+aahung@users.noreply.github.com> * apply review comments * typo * read using utf-8 * create and user a safe version of the save_config method * apply review comments * rename _get_command_name to _get_command_names * don't save generated ARNs for now, will save during init * Revert "don't save generated ARNs for now, will save during init" This reverts commit d184e164022d9560131c62a826436edbc93da189. * Notify the user to rotate periodically rotate the IAM credentials * typo * Use AES instead of KMS for S3 SSE * rename Ecr to ECR and Iam to IAM * Grant lambda service explicit permissions to thhe ECR instead of relying on giving this permissions on ad-hoc while creating the container images Co-authored-by: Chris Rehn <crehn@outlook.com> Co-authored-by: _sam <3804518+aahung@users.noreply.github.com> * sam pipeline init command (aws#2831) * sam pipeline init command * apply review comments * apply review comments * display a message that we have successfully created the pipeline configuration file(s). * doc typo * Let 'sam pipeline init' prefills pipeline's infrastructure resources… (aws#2894) * Let 'sam pipeline init' prefills pipeline's infrastructure resources' values from 'sam pipeline bootstrap' results. * save bootstrapped sateg region * make black happy * exclude non-dict keys from samconfig.get_env_names method. * Rename the pipeline 'Stage' concept to 'Environment' (aws#2908) * Rename the pipeline 'Stage' concept to 'Environment' * typo * Rename --environment-name argument to --environment * Sam pipelines ux rename ecr repo to image repository (aws#2910) * Rename ecr-repo to image-repository * UT Fixes * typo * typo * feat: Support creating pipeline files directly into . without hooks (aws#2911) * feat: Support creating pipeline files directly into . without hooks * Integration test for pipeline init and pipeline bootstrap (aws#2841) * Expose Environment._get_stack_name for integ test to predict stack name * Add integ test for pipeline bootstrap * Add init integ test * small UX improvements: (aws#2914) * small UX improvements: 1. show a message when the user cancels a bootstrapping command. 2. Don't prompt for CI/CD provider or provider templates if there is only one choice. 3. Make PipelineFileAlreadyExistsError a UserError. 4. use the Colored class instead of fg='color' when prompting a colored message. 5. Fix a bug where we were not allowing empty response for not required questions. * Fix Integration Test: We now don't ask the user to select a provider's pipeline template if there is only one * Add docs for PipelineFileAlreadyExistsError * make black happy * Sam pipelines s3 security (aws#2975) * Deny non https requests for the artifacts S3 bucket * enable bucket serverside logging * add integration tests for artifacts bucket SSL-only requests and access logging * typo * Ensure the ArtifactsLoggingBucket denies non ssl requests (aws#2976) * Sam pipelines ux round 3 (aws#2979) * rename customer facing message 'CI/CD provider' to 'CI/CD system' * add a note about what 'Environment Name' is during the pipeline bootstrap guided context * Apply suggestions from code review typo Co-authored-by: Chris Rehn <crehn@outlook.com> Co-authored-by: Chris Rehn <crehn@outlook.com> * let pipeline IAM user assume only IAM roles tagged with Role=pipeline-execution-role (aws#2982) * Adding AWS_ prefix to displayed out. (aws#2993) Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Add region to pipeline bootstrap interactive flow (aws#2997) * Ask AWS region in bootstrap interactive flow * Read default region from boto session first * Fix a unit test * Inform write to pipelineconfig.toml at the end of bootstrap (aws#3002) * Print info about pipelineconfig.toml after resources are bootstrapped * Update samcli/commands/pipeline/bootstrap/cli.py Co-authored-by: Chris Rehn <crehn@outlook.com> Co-authored-by: Chris Rehn <crehn@outlook.com> * List detected env names in pipeline init when prompt to input the env name (aws#3000) * Allow question.question can be resolved using key path * Pass the list of env names message (environment_names_message) into pipeline init interactive flow context * Update samcli/commands/pipeline/init/interactive_init_flow.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Fix unit test (trigger pr builds) * Fix integ test * Fix integ test Co-authored-by: Chris Rehn <crehn@outlook.com> * Adding account id to bootstrap message. (aws#2998) * Adding account id to bootstrap message. * adding docstring * Addressing PR comments. * Adding unit tests. * Fixing unit tests. Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Cfn creds fix (aws#3014) * Removing pipeline user creds from cfn output. This maintains same user exp. Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Ux bootstrap revamp 20210706 (aws#3021) * Add intro paragraph to bootstrap * Add switch account prompt * Revamp stage definition prompt * Revamp existing resources prompt * Revamp security prompt * Allow answers to be changed later * Add exit message for bootstrap * Add exit message for bootstrap (1) * Add indentation to review values * Add "Below is the summary of the answers:" * Sweep pylint errors * Update unit tests * Update samcli/commands/pipeline/bootstrap/guided_context.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update samcli/commands/pipeline/bootstrap/guided_context.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update samcli/commands/pipeline/bootstrap/guided_context.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update samcli/commands/pipeline/bootstrap/guided_context.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update samcli/commands/pipeline/bootstrap/guided_context.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update samcli/commands/pipeline/bootstrap/guided_context.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update samcli/commands/pipeline/bootstrap/guided_context.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update samcli/commands/pipeline/bootstrap/guided_context.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update samcli/commands/pipeline/bootstrap/cli.py Co-authored-by: Chris Rehn <crehn@outlook.com> * Update unit tests * Add bold to other literals Co-authored-by: Chris Rehn <crehn@outlook.com> * Adding account condition for CFN execution role. (aws#3027) Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * pipeline UX revamp 20210707 (aws#3031) * Allow running bootstrap inside pipeline init * Select account credential source within bootstrap * Add bootstrap decorations within pipeline init * Removing ip range option from bootstrap. (aws#3036) * Removing ip range option from bootstrap. * Fixing unit test from UX PR. Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Fix toml file incorrect read/write in init --bootstrap (aws#3037) * Temporarily removing account fix. (aws#3038) Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Rename environment to stage (aws#3040) * Improve account source selection (aws#3042) * Fixing various cosmetics UX issues with pipeline workflow. (aws#3046) * Fixing credential to credentials * Forcing text color to yellow. * Adding new line after stage diagram. * Adding extra line after checking bootstrap message. * Renaming config -> configuration * account source -> credential source * Removing old message. * Fixing indentation in list. * Fixing bunch of indentation. * fixing f string Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Auto skip questions if stage detected (aws#3045) * Autofill question if default value is presented * Allow to use index to select stage names (aws#3051) * Updating message when bootstrap stages are missing. (aws#3058) * Updating message when bootstrap stages are missing. * Fixing indendation Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Fixing bootstrap integ tests. (aws#3061) * Fixing bootstrap integ tests. * Cleaning up some integ tests. * Using environment variables when running integ test on CI. * Using expression instead of full loop. * Adding instruction to use default profile on local. Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Fix bootstrap test region (#3064) * Fix bootstrap region in integ test * Fix regions in non-interactive mode as well * Add more pipeline init integ test (aws#3065) * Fix existing pipeline init integ test * Add more pipeline init integ tests * Config file bug (aws#3066) * Validating config file after bootstrap stack creation. * Validating config file after bootstrap. Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> * Fix pipeline init integ test because of pipelineconfig file exists (aws#3067) * Make stage name randomized to avoid race condition among multi canary runs (aws#3078) * Load number of stages from pipeline template (aws#3059) * Load number of stages from templates * Rename variable and add debug log * Add encoding to open() * Allow roles with Tag aws-sam-pipeline-codebuild-service-role to assume PipelineExecutionRole (aws#2950) * pipeline init UX: Ask to confirm when file exists (aws#3079) * Ask to confirm overriding if files already exist, or save to another directory * Add doc links (aws#3087) * Adding accidentally removed tests back. (aws#3088) Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com> Co-authored-by: elbayaaa <72949274+elbayaaa@users.noreply.github.com> Co-authored-by: Chris Rehn <crehn@outlook.com> Co-authored-by: Ahmed Elbayaa <elbayaaa@amazon.com> Co-authored-by: Tarun <c2tarun@users.noreply.github.com> Co-authored-by: Tarun Mall <tarun@amazon.noreply.github.com>
From the documentation for
aws rds download-db-log-file-portionActual results show the data is being truncated.
CLI Version
Related issues:
#2745
#2268
Debug output with log information stripped
aws rds download-db-log-file-portion --db-instance-identifier xxxx --log-file-name error/xxxx --starting-token 0 --debug 2> out
The text was updated successfully, but these errors were encountered: