Support for Civitai.com

[Devicetron]

is there a way to download images from civitai.com with gallery-dl?

[mikf]

If it's not listed in docs/supportedsites, then no.
At least not until someone adds support for this site.

[Devicetron]

my skill level is very low so couldnt make it work, hopefully someone will take a look at their api https://github.com/civitai/civitai/wiki/REST-API-Reference

[Dark-Obsidian]

my skill level is very low so couldnt make it work, hopefully someone will take a look at their api https://github.com/civitai/civitai/wiki/REST-API-Reference

I am pretty new to API calls but this PowerShell script should do what you want...

Get-CivitAiImages.ps1
=====================

Function Get-CivitAiImages {

    Param(
        [Parameter(Mandatory = $true, Position = 0)][string]$Username
    )

    $BaseFolder = "<<SET YOUR BASE DOWNLOAD FOLDER HERE>>"

    $OutFolder = "$BaseFolder\$Username"
    $JSONPath = "$OutFolder\$Username.json"
    If (!(Test-Path $OutFolder)){ New-Item -Path $OutFolder -ItemType Directory }
    Set-Location $OutFolder

    $ImageJSON = @()
    $NextCursor = 1
    
    do {
        $ThisCursor = $NextCursor
        $URL = "https://civitai.com/api/v1/images?username=$Username&cursor=$ThisCursor"
        curl $URL -H "Content-Type: application/json" -X GET -o $JSONPath -s
        try{     $JSON = Get-Content -LiteralPath $JSONPath -Raw | ConvertFrom-Json }
          catch{ $JSON = Get-Content -LiteralPath $JSONPath -Raw | ConvertFrom-Json -AsHashtable }
        $ImageJSON += $JSON.items
        $NextCursor = $JSON.metadata.nextCursor
    } while (
        $null -ne $NextCursor
    )

    $Global:i = 0
    $ItemCount = $ImageJSON.Count
    $Activity = "Downloading images... "

    $ImageJSON | ConvertTo-Json -Depth 10 | Out-File -LiteralPath $JSONPath -Force
    for ($i = 0; $i -lt $ItemCount; $i++) {
        $ImageURL = $ImageJSON[$i].url
        curl $ImageURL -O -s -R -C -

        $Global:i++; $Percent = 100 / $ItemCount * $Global:i
        $Status = " {0:n1}%:   {1} of {2}" -f $Percent, $Global:i, $ItemCount
        Write-Progress -Activity $Activity -Status $Status -PercentComplete $Percent
    }

    Write-Progress -Activity $Activity -Completed
}

✅ It will download all the images for a given user, as well as a .json containing all the metadata for all images.
❌ It is completely separate to gallery-dl and so does not have any of gallery-dl's options or features

---

USAGE
=====

> pwsh                                  | open PowerShell
PS> .\Get-CivitAiImages.ps1             | load the function
PS> Get-CivitAiImages "user_to_dl"      | download images for user_to_dl

[Hrxn]

Deduction of points for using curl, not PowerShell's own functions 😄

[Dark-Obsidian]

Lol... Hey, don't look at me for that bit - I was just extending the CivitAi example 😇

Joking aside though...

• I don't have much experience with curl (and even less of Invoke-WebRequest) <-- am guessing off the top of my head that's the native cmdlet
• Is Invoke-WebRequest better, generally?
• The script could definitely use improvement (for example: it doesn't parse/apply any of the metadata from the .json file) but hoped it would help a few people out as a 'stop-gap' solution until gallery-DL supports civitAi.com officially (...assuming this is on the todo-list somewhere down the line)
• That said, if anyone wants to take this script as a starting point and enhance / extend it, please definitely feel free! 👍🏼

[Hrxn]

Yeah, Invoke-WebRequest would be the correct command for downloading stuff..

Although there's also Invoke-RestMethod, which is useful for dealing with APIs, for example. Simply put, if anything returns JSON, you can use Invoke-RestMethod (although it's not necessary. It's just a bit shorter, because you can save one step, basically).

It's not that Invoke-WebRequest is "better" (what does better even mean in that context?), it's simply native to PowerShell whereas curl would be counted as an external dependency. Which is not really a big deal, since current versions of Windows 10 as well as Windows 11 ship curl by default (although not the latest release version of curl, obviously), but still, it is usually considered good practice to avoid external dependencies if it's feasible to avoid them..

And Invoke-WebRequest is also very flexible, I've never seen a scenario where this would be a limiting factor, compared to curl, or whatever..

And if it should be necessary, like for some pretty exotic stuff, you can just use the entirety of the .NET platform (i.e. System.Net.Http) directly from PowerShell, since it is built on .NET too.