Get-CosmosDbDocument doesn't get all documents

[Panzerbjrn]

Issue

• [PSVersion - 5.1.18362.145 ] Version of PowerShell you're using
• [Console Host] PowerShell host you're using (eg. Console Host, ISE, Visual Studio)
• [Windows 10] Operating system you're running
• [3.7.0 - CosmosDB] Version of CosmosDB PowerShell Module you're using (use Get-Module -Name CosmosDB)

Hi all, when I use Get-CosmosDbDocument, example Get-CosmosDbDocument -Context $CosmosDbContext -CollectionId $CosmosCollection, I only get 731 documents returned (out of 36255).
I tried the parameter -MaxItemCount 800, but still only received 731 documents.
If I use | select -Skip 500 I get 231, which makes sense, but it was worth trying.

Is this a bug? Or intentional limitation? How would I get all documents when -MaxItemCount doesn't seem to do anything?
I also don't get any errors, which seems odd.
Thanks.

[PlagueHO]

Hi @Panzerbjrn - the Get-CosmosDbDocument function when not specifying a query and not specifying an ID will call the List Documents REST API (https://docs.microsoft.com/en-us/rest/api/cosmos-db/list-documents). This API docs doesn't identify any limitations on the REST API, however, there are some documented here:
https://docs.microsoft.com/en-us/azure/cosmos-db/concepts-limits#per-request-limits

Specifically: The Maximum response size (for example, paginated query) is 4MB. Therefore, I suspect those 731 documents are just under 4MB and so CosmosDB is cutting it off there.

So what you want to do is use a continuation token to retrieve the documents following. You could do this in a while loop reasonably easily (I haven't tested this code but it should be correct):

$documentsPerRequest = 20
$continuationToken = $null
$documents = $null

do {
    $responseHeader = $null
    $getCosmosDbDocumentParameters = @{
        Context = $cosmosDbContext
        CollectionId = 'MyNewCollection'
        MaxItemCount = $documentsPerRequest
        ResponseHeader = ([ref] $responseHeader)
    }

    if ($continuationToken) {
        $getCosmosDbDocumentParameters += {
            ContinuationToken = $continuationToken
        }
    }
    $documents += Get-CosmosDbDocument @getCosmosDbDocumentParameters 
    $continuationToken = [System.String] $responseHeader.'x-ms-continuation'
} while (-not [System.String]::IsNullOrEmpty($continuationToken))

Does that make sense?

I should probably add the above code as a sample into the README (after I test it 😁).

[Panzerbjrn]

Yessssssssssssssssssssssssssssssssssssssss, that makes sense and mostly works. I have to admit that I don't quite get how the continuation token works, but it does...

Oddly, I was getting this error.

A hash table can only be added to another hash table.
At line:15 char:9
+         $GetCosmosDbDocumentParameters += {

So I changed
Oddly, I was getting this error.

$getCosmosDbDocumentParameters += {
            ContinuationToken = $continuationToken
        }

to $GetCosmosDbDocumentParameters.ContinuationToken = $ContinuationToken and it worked.
Incidentally, that is my personally preferred way of doing it when there is just one parameter to change.
Thanks for your help :-D

[PlagueHO]

Opening this again so I can add this example to the documentation.