M: Can I tell the gallery-dl utility to add webm and mp4 files

G: Yes, you can configure gallery-dl to download specific file types, including webm and mp4, by adjusting its configuration file (config.json). Here’s how:

  1. Locate or create the config.json file. The file is typically found in:

    • Linux: ~/.config/gallery-dl/config.json
    • Windows: %APPDATA%\gallery-dl\config.json
  2. Add or modify the format section to include the desired file extensions. For example:

{
    "extractor": {
        "base-directory": "~/Downloads",
        "archive": "archive.sqlite3",
        "filename": "{category}/{author}/{id}.{extension}",
        "postprocessors": [
            {
                "name": "metadata"
            }
        ]
    },
    "format": {
        "file": {
            "extensions": ["webm", "mp4"]
        }
    }
}
  • Restart or re-run gallery-dl to apply the changes.

  • Command-Line Alternative

    If you want to filter downloads by file type on a per-run basis, use the --filter option:

    gallery-dl "URL" --filter "extension in ('webm', 'mp4')"

    This ensures only webm and mp4 files are downloaded.

    M: Can I add a delay between each file downloaded of 2 seconds?

    G: Yes, you can configure gallery-dl to add a delay between each downloaded file by using the sleep option in the configuration file or specifying it in the command line.

    Configuring Delay in config.json

    Add the sleep-request option under the extractor section of your config.json file:

    {
        "extractor": {
            "sleep-request": 2.0
        }
    }

    This setting introduces a 2-second delay between each request made by gallery-dl.

    Command-Line Alternative

    If you want to set the delay on a per-run basis, use the --sleep-request option:

    gallery-dl --sleep-request 2 "URL"

    Notes

    M: Sleep delay isn’t working

    G: If --sleep-request or "sleep-request" in config isn't working as expected, here are likely reasons and correct alternatives:


    🔧 Important Clarification


    ✅ Solution: Use sleep instead of sleep-request

    sleep adds a delay between each download, which is what you want.

    Add to config.json:

    {
        "extractor": {
            "sleep": 2.0
        }
    }

    This tells gallery-dl to wait 2 seconds between each item.

    Or use via command-line:

    gallery-dl --sleep 2 "URL"

    ✅ Extra: Combine with --verbose to verify sleep

    Run with --verbose to confirm it's pausing:

    gallery-dl --sleep 2 --verbose "URL"

    You’ll see log lines indicating sleep time between items.


    Let me know if it still skips the delay after that.

    M: i swear i asked you about this, but let's start over because i lost all my data... i'm trying to run gallery-dl, configuring it to replace uncommon/problematic characters with underscores, delay a couple seconds to prevent server overloads, and automatically number things including leading zeros. like, let's say i wanted to use a file named 'list.txt' containing URLs like "https://boards.4chan.org/w/thread/2207908/ranma-12-papes" to get a decent wallpaper collection; check online, then walk me through what i need to do.

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: reasoning_recap; content_type=reasoning_recap]

    G: TL;DR — Yes. Put your URLs in list.txt, create a config.json that (1) replaces ugly/path-breaking characters with _, (2) sleeps ~2s between files, and (3) zero-pads numbers in filenames. Then run:

    gallery-dl -i list.txt

    Details below, including Fedora install and a ready-to-paste config.


    1) Install (Fedora-friendly)

    # Fedora repo
    sudo dnf install gallery-dl
    # …or latest via pipx (keeps it isolated)
    python3 -m pip install --user pipx && pipx ensurepath
    pipx install gallery-dl

    Where config files live (any of these works): ~/.config/gallery-dl/config.json, ~/.gallery-dl.conf, or /etc/gallery-dl.conf. citeturn3search8

    2) Put your URLs in a file

    Create list.txt (one URL per line), e.g.:

    https://boards.4chan.org/w/thread/2207908/ranma-12-papes

    Then you can run all of them with -i list.txt (reads from file/stdin). citeturn0search3turn0search8

    3) Minimal, drop-in config.json

    This gives you:

    {
      "base-directory": "~/Downloads/gallery-dl",
      "extractor": {
        "path-restrict": {
          " ": "_",
          "/": "_",
          "\\": "_",
          "|": "_",
          ":": "_",
          "*": "_",
          "?": "_",
          "\"": "_",
          "<": "_",
          ">": "_"
        },
        "path-remove": "\u0000-\u001f\u007f",
        "sleep": 2.0,
        "sleep-request": 0.0,
        "sleep-extractor": 0.0,
    
        "#": "Default filename: site/post-or-thread/NNN.ext with zero padding",
        "filename": "{num:0>3}.{extension}",
        "directory": ["{category}", "{title|id}"]
      }
    }

    Why these options work:

    4) Run it

    # Basic: read URLs from list.txt and go
    gallery-dl -i list.txt
    
    # Show what keys you can use in filenames for a given URL (super helpful)
    gallery-dl -K "https://boards.4chan.org/w/thread/2207908/ranma-12-papes"

    -K/--list-keywords prints the variables available for that extractor (so you can refine your naming). citeturn2view0

    Notes specific to your goals

    If you want, I can tune the filename to include board name, thread ID, post time, and keep the zero-padded {num} so your wallpapers sort perfectly.

    [Attachment reference unresolved: file-names]

    [Attachment reference unresolved: file-naming]

    [Attachment reference unresolved: file-album]

    M: additions: i want any image tags to be saved to the image (or if not there, then to a separate file); the first item in the file name should be (with leading zeros if necessary) the post number, for absolute comparison; and tell me if there's a way to avoid redundancy - like, if i was doing an image search at danbooru, and wanted to first get everything with the tags "xenosaga -short_hair" and then everything with the tag "xenosaga", but not download the stuff already acquired, is there a built in method of doing this?

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: reasoning_recap; content_type=reasoning_recap]

    G: TL;DR — Yes. You can 1) embed a 2s delay between files, 2) sanitize filenames to underscores, 3) start filenames with a zero-padded post ID, 4) save tags (JSON sidecar and/or a plain “tags.txt”), and 5) avoid re-downloading when you run multiple Danbooru queries by using the download archive. Run with -i list.txt and the config below.


    1) Drop-in config.json

    Put this at ~/.config/gallery-dl/config.json:

    {
      "base-directory": "~/Downloads/gallery-dl",
    
      "extractor": {
        "sleep": 2.0,
    
        "path-restrict": "ascii",
        "path-replace": "_",
    
        "archive": "~/Downloads/gallery-dl/archive.sqlite3",
    
        "filename": "{num|id:0>9}.{extension}"
      },
    
      "postprocessors": [
        { "name": "metadata", "mode": "json", "event": "after" },
        { "name": "metadata", "mode": "tags", "event": "after" }
      ]
    }

    What this does:

    Tip: For Danbooru you’ll get rich tags in metadata; 4chan won’t have image tags, but you’ll still get per-post info in the JSON. citeturn0search15


    2) Use it with your list.txt

    Your list.txt might contain:

    https://boards.4chan.org/w/thread/2207908/ranma-12-papes

    Run:

    gallery-dl -i list.txt

    If you prefer CLI flags instead of config for some parts:

    gallery-dl -i list.txt \
      --sleep 2 \
      --download-archive ~/Downloads/gallery-dl/archive.sqlite3

    The archive ensures that when you later run something broader (e.g., xenosaga -short_hair first, then xenosaga), already-seen IDs are skipped, so no duplicates are downloaded. citeturn0search20


    3) Make the filename exactly how you want

    You can confirm the available keys per URL with:

    gallery-dl -K "https://boards.4chan.org/w/thread/2207908/ranma-12-papes"
    gallery-dl -K "https://danbooru.donmai.us/posts?tags=xenosaga"

    (-K/--list-keywords shows the exact fields you can use.) citeturn0search16


    4) (Optional) Embed tags into images with ExifTool

    If you want the tags inside the image metadata (instead of sidecars), use the exec postprocessor to call exiftool after each download. Example sketch:

    {
      "postprocessors": [
        { "name": "metadata", "mode": "json", "event": "after" },
        {
          "name": "exec",
          "event": "after",
          "command": [
            "exiftool",
            "-overwrite_original",
            "-charset", "filename=utf8",
            "-XMP-dc:Subject<=${metadata[tags]|[]|join(',',_)}",
            "{filepath}"
          ]
        }
      ]
    }

    (The pattern above assumes tags are available; tune the expression to your preference. See community examples of using exec + ExifTool.) citeturn0search8turn0search13


    5) Avoiding redundancy across multiple runs (Danbooru example)


    6) Sanitize exactly how you like (optional advanced)

    If you prefer a custom mapping instead of blanket ASCII stripping, use a dict mapping in path-restrict (e.g., turn spaces and any specific symbols into _). Example:

    "extractor": {
      "path-restrict": {
        " ": "_",
        "/": "_",
        "\\": "_",
        "\"": "_",
        "<": "_",
        ">": "_",
        "|": "_",
        ":": "_",
        "*": "_",
        "?": "_"
      },
      "path-remove": "\u0000-\u001f\u007f",
      "path-replace": "_"
    }

    See maintainer guidance on path-restrict strategies. citeturn0search4


    If you want, tell me whether you’re prioritizing 4chan or Danbooru first and I’ll tailor the exact filename/directory to include board, thread, or tag slices while keeping the post ID first for strict sorting.

    [Attachment reference unresolved: file-names]

    M: one more thing - what if there's a particular string i want to append to every url? like, with danbooru, if i wanted to append "+-pencil" to prevent any pictures with pencils to every search, could i do that simply?

    G: Good question — there isn’t a single “append this to every URL” switch in gallery-dl, but there are two easy ways to get the effect you want for Danbooru (or other query-string–based extractors):


    1. Pre-process the URL list

    Simplest: edit or generate list.txt so every query already has the suffix you want, e.g.:

    https://danbooru.donmai.us/posts?tags=xenosaga+-pencil
    https://danbooru.donmai.us/posts?tags=xenosaga+-short_hair+-pencil

    If you don’t want to edit by hand, you can wrap it with a shell one-liner:

    # Take list.txt, append "+-pencil" to every line, pipe into gallery-dl
    sed 's/$/+-pencil/' list.txt | gallery-dl -i -

    This way gallery-dl never sees the unmodified URL.


    2. Use --exec as a wrapper

    Another trick is to preprocess on the fly: you can run gallery-dl in a loop where you transform the URL before handing it over. For example in Bash:

    while read -r url; do
      gallery-dl "${url}+-pencil"
    done < list.txt

    That appends +-pencil every time without altering list.txt.


    3. Config-file (indirect)

    gallery-dl’s config doesn’t have a built-in “always append this query string,” but if you want to bake it in permanently for Danbooru, you can wrap gallery-dl with a little shell alias:

    alias danbooru-dl='gallery-dl "https://danbooru.donmai.us/posts?tags="

    and then supply only the base tags. Or create a tiny wrapper script that adds your +-pencil automatically before calling gallery-dl.


    Recommendation

    For your specific case (exclude pencil in every search), the sed trick is the least effort — it works for any existing list.txt, and you don’t have to modify the config. If you’re going to do it often, a wrapper script is better so you never forget.


    👉 Do you want me to sketch out a Bash wrapper (like gdl-danbooru) that you can drop in ~/bin and then always type gdl-danbooru xenosaga to get the right URL with +-pencil appended? That way you never touch the raw list.

    M: what if i want to use a local custom json for a particular directory

    G: gallery-dl can use a local, directory-specific custom JSON config — you just need to invoke it correctly.
    There is no automatic “per-directory config discovery”, but you can easily achieve the effect.

    Below are all workable methods, from cleanest to hackiest.


    ✔️ Method 1: Use -c ./config.json inside that directory

    This is officially supported.

    If you have:

    /home/user/wallpapers/
        custom-config.json
        list.txt

    Then you run:

    cd /home/user/wallpapers
    gallery-dl -c custom-config.json -i list.txt

    gallery-dl will use that JSON only for this run.
    This is the cleanest, simplest, most reliable method.


    ✔️ Method 2: Create a wrapper script in that directory

    Example: /home/user/wallpapers/gdl.sh

    #!/bin/bash
    gallery-dl -c "$PWD/custom-config.json" "$@"

    Make executable:

    chmod +x gdl.sh

    Now simply use:

    ./gdl.sh -i list.txt

    This makes the directory “self-contained”.


    ✔️ Method 3: Use a directory-specific alias

    If you always run gallery-dl inside the directory, create an alias in ~/.bashrc or ~/.zshrc:

    alias gdlw='gallery-dl -c /home/user/wallpapers/custom-config.json'

    Then:

    cd /home/user/wallpapers
    gdlw -i list.txt

    ✔️ Method 4: Use an environment variable to force config

    If you want gallery-dl to always load your custom JSON when run from that shell:

    export GALLERY_DL_CONFIG="/home/user/wallpapers/custom-config.json"

    Then run:

    gallery-dl -i list.txt

    Note: This affects the whole shell session, not just the directory.


    ✔️ Method 5 (Advanced): Make gallery-dl auto-detect a directory-local config

    You can fake “per-directory config auto-loading” with a tiny launcher script:

    Create /usr/local/bin/gallery-dl-local:

    #!/bin/bash
    
    # If config.json exists in cwd, use it.
    if [[ -f "./config.json" ]]; then
        gallery-dl -c ./config.json "$@"
    else
        gallery-dl "$@"
    fi
    sudo chmod +x /usr/local/bin/gallery-dl-local

    Use it instead of gallery-dl:

    gallery-dl-local -i list.txt

    Now any directory can have its own config.json, and it will be auto-used.


    ✔️ Summary (most important takeaways)


    If you want, I can build you a ready-made directory-local config loader that also:

    Just tell me.

    M: i tried piecing together things again step by step, but likely missed a comma or something. can you please review and give me a functional version? check online if you need to - but all i need is syntax correction, no other modification.

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: reasoning_recap; content_type=reasoning_recap]

    G: Here’s a syntactically valid version of your config, with only the minimum fixes:

    {
      "base-directory": "~/Downloads/gallery-dl",
      "extractor": {
        "path-restrict": {
          " ": "_",
          "/": "_",
          "\\": "_",
          "|": "_",
          ":": "_",
          "*": "_",
          "?": "_",
          "\"": "_",
          "<": "_",
          ">": "_"
        },
        "path-remove": "\u0000-\u001f\u007f",
        "path-replace": "_",
        "sleep": 2.0,
        "sleep-request": 0.0,
        "sleep-extractor": 0.0,
        "archive": "~/Downloads/gallery-dl/archive.sqlite3",
        "filename": "{num|id:0>9}.{extension}",
        "#": "Default filename: site/post-or-thread/NNN.ext with zero padding",
        "directory": ["{category}", "{title|id}"]
      },
      "postprocessors": [
        { "name": "metadata", "mode": "json", "event": "after" },
        { "name": "metadata", "mode": "tags", "event": "after" },
        {
          "name": "exec",
          "event": "after",
          "command": [
            "exiftool",
            "-overwrite_original",
            "-charset",
            "filename=utf8",
            "-XMP-dc:Subject<=${metadata[tags]|[]|join(',',_)}",
            "{filepath}"
          ]
        }
      ]
    }

    You can drop this in as config.json and gallery-dl should parse it without complaints.

    M: okay, currently it's trying to run, but every item translates the filename to 'None.[ext]', where [ext] is gif or whatever, and then skips everything but the first item since there's already a file with that name.

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    M: i think you may have started to think too deeply. take a deep breath and tell me what you think this json configuration file is supposed to do, based on the configuration i uploaded and what you remember from this silo; it was started a while ago, so you might not actually have everything i think you have to work with. remember, this is for gallery-dl and you have permission to go online if necessary.

    G: Understood — stepping back, clearing the slate, and treating this as “What is this config intended to do?”
    Not “what is wrong,” not “how to fix it,” but “what was the purpose of this configuration based on what you uploaded and what we discussed earlier?”

    Here is the high-level interpretation, without overthinking:


    What your configuration file is supposed to do

    Based on the JSON you uploaded and the earlier goals in this silo, this config is intended to achieve five major behaviors:


    1. Sanitize filenames by replacing problematic characters with underscores

    Your path-restrict block:

    "path-restrict": {
      " ": "_",
      "/": "_",
      "\\": "_",
      "|": "_",
      ":": "_",
      "*": "_",
      "?": "_",
      "\"": "_",
      "<": "_",
      ">": "_"
    }

    This means:

    Purpose:
    Make filenames and directories safe and consistent everywhere (Linux, Windows, future archives, etc.).


    2. Remove invisible / control characters

    "path-remove": "\u0000-\u001f\u007f"

    This strips ASCII control characters.

    Purpose:
    Avoid bizarre rare characters appearing in filenames (common with scraped text).


    3. Enforce a uniform delay between downloaded files

    "sleep": 2.0

    This is the between-files delay, what you explicitly wanted to prevent server hammering.

    Purpose:
    Slow down downloads ~2 seconds per file so you don’t accidentally hammer services (4chan, Danbooru).


    4. Name files with a zero-padded post identifier as the first filename element

    "filename": "{num|id:0>9}.{extension}"

    This means:

    Purpose:


    5. Keep a download archive so re-running searches doesn’t download duplicates

    "archive": "~/Downloads/gallery-dl/archive.sqlite3"

    This is specifically for your earlier question:

    If I download “xenosaga -short_hair” and then “xenosaga”, can I avoid overlap?

    Purpose:
    Record every downloaded file’s ID so future runs skip duplicates.


    6. Save metadata — including image tags — into separate files and/or embed them

    Your postprocessors block asks for three things:

    a) Structured metadata (JSON)

    { "name": "metadata", "mode": "json", "event": "after" }

    Produces:
    000001234.json
    Containing everything gallery-dl knows (IDs, tags, timestamps, etc.).

    b) Tag list

    { "name": "metadata", "mode": "tags", "event": "after" }

    Produces:
    000001234.tags
    A simple newline-separated tag list.

    c) Optional ExifTool tag embedding

    "name": "exec",
    "command": [ "exiftool", ..., "{filepath}" ]

    This attempts to write tags directly into the image’s XMP metadata.

    Purpose:


    7. Store files in a site-/thread-/category-based directory structure

    "directory": ["{category}", "{title|id}"]

    Meaning:

    Purpose:
    Organize downloads sanely.


    So the intended behavior is:

    “Download everything safely, delay politely, name everything by post-number, guarantee no duplicates across searches, sanitize filenames, save tags separately, embed metadata if possible, and organize output into human-readable folders.”

    This matches exactly what you’ve been aiming for in this silo:


    ✔ Where the problem came from (“None.ext”) — without going into fix mode

    Just describing intention, but noting the mismatch:

    Your filename rule:

    "{num|id:0>9}.{extension}"

    assumes every extractor provides either num or id.

    If neither exists for a particular site or sub-item (thumbnails, inline links, removed posts, etc.),
    gallery-dl substitutes None.

    So:
    Your config is correct in purpose — the site simply didn’t provide the fields your rule depended on.

    (And since every file becomes None.ext, everything after the first gets skipped due to name collision.)


    If you want, now that your intentions are clarified cleanly, I can generate a corrected version that:

    Just tell me to proceed.

    M: okay, keeping things isolated instead of trying to fix every possible variation, tell me how - if it is possible - to modify the current version of the config file to follow the current rule when the information is available - but to use default information if it is not; if this is not possible, tell me so. then, try and tell me why when i was trying to update gallery-dl with dnf (sudo dnf update gallery-dl), it didn't work - like, did i use bad syntax, or is the utility discontinued, or..?

    G: Below is the clean, direct answer, keeping everything isolated, no overreach, and no modifying unrelated parts.


    1. Can your current filename rule fall back to a default when {num} and {id} are missing?

    Short answer:
    Yes — but only if you supply the fallback yourself.
    gallery-dl does not have a built-in automatic “default variable” for filenames.

    The syntax you used:

    {num|id:0>9}

    means:

    But if both are missing, gallery-dl inserts the Python None object → "None".

    To avoid this, you must explicitly give a fallback value:

    Working version with fallback

    "{num|id|filename:0>9}.{extension}"

    OR even safer, if you want a guaranteed numeric sequence:

    "{num|id|index:0>9}.{extension}"

    Where:

    Result:

    Thus:

    YES, it is possible to modify your config so it ONLY uses your desired rule when available, and a safe default otherwise.

    This is the simplest, cleanest, and officially supported fallback mechanism.


    2. Why did sudo dnf update gallery-dl not work?

    Because dnf update X is not a valid command to update a single package.

    Fedora / DNF syntax is not like apt.

    ✔ Correct syntax:

    sudo dnf upgrade
    sudo dnf upgrade gallery-dl

    ✔ Bonus: another reason it often fails

    Fedora’s default repo sometimes lags behind the latest gallery-dl release.

    If the version in the repo is already “the latest according to Fedora,” then:

    dnf upgrade gallery-dl

    will simply respond:

    Nothing to do.

    This does NOT mean gallery-dl is discontinued — only that:


    If you need the newest gallery-dl on Fedora

    Use pipx, which cleanly isolates it:

    pipx install gallery-dl
    pipx upgrade gallery-dl

    This avoids the lag between upstream releases and Fedora packages.


    ✔ Summary

    Filename fallback?

    Yes. Use:

    "{num|id|index:0>9}.{extension}"

    You’ll never see None.ext again.

    DNF update failed why?

    Because dnf update package isn’t correct syntax,
    and Fedora’s gallery-dl packages often lag behind upstream.

    If you want, I can generate a clean final config with the fallback included and nothing else changed.

    M: (base) [david@new-host-1 ____gallery]$ pip upgrade gallery-dl ERROR: unknown command "upgrade" (base) [david@new-host-1 ____gallery]$ pip3 upgrade gallery-dl ERROR: unknown command "upgrade" (base) [david@new-host-1 ____gallery]$ pipx upgrade gallery-dl bash: pipx: command not found... Similar command is: 'pip' (base) [david@new-host-1 ____gallery]$ pip update gallery-dl ERROR: unknown command "update" (base) [david@new-host-1 ____gallery]$ sudo dnf upgrade gallery-dl Updating and loading repositories: Repositories loaded. Failed to resolve the transaction: No match for argument: gallery-dl You can try to add to command line: --skip-unavailable to skip unavailable packages (base) [david@new-host-1 ____gallery]$

    G: You didn’t do anything wrong — this is just the exact intersection of three different systems (pip, pipx, and DNF) each behaving correctly but differently from what their names suggest.

    Below is the straight explanation, no fluff, no glossing:


    ✔ WHY pip upgrade / pip update DOESN’T WORK

    Because pip does not have upgrade/update subcommands.
    The only valid way to upgrade a pip-installed package is:

    pip install --upgrade gallery-dl

    or:

    pip3 install --upgrade gallery-dl

    That’s it.
    pip upgrade will always throw:

    ERROR: unknown command "upgrade"

    ✔ WHY pipx DOESN’T WORK

    You got this:

    bash: pipx: command not found

    That means pipx is not installed.

    Install it like this:

    sudo dnf install pipx
    pipx ensurepath

    Then:

    pipx install gallery-dl
    pipx upgrade gallery-dl

    pipx is optional, but by far the cleanest way to run command-line tools isolated from system packages.


    ✔ WHY dnf upgrade gallery-dl FAILED

    Your error:

    No match for argument: gallery-dl

    This means Fedora’s repos on your system do not contain gallery-dl at all.

    On Fedora, gallery-dl is usually packaged as:

    python3-gallery-dl

    Try:

    sudo dnf install python3-gallery-dl

    To upgrade it later:

    sudo dnf upgrade python3-gallery-dl

    If that still says “No match,” then your current Fedora version / repo set simply doesn’t include it.


    ✔ SO WHICH ONE DID YOU INSTALL ORIGINALLY?

    Given the errors you saw, it is almost certain that:

    Check:

    which gallery-dl

    You’ll probably see something like:

    /home/david/miniconda3/bin/gallery-dl

    or:

    ~/.local/bin/gallery-dl

    Either way → DNF will never see it, because it was not installed through the Fedora package manager.


    ✔ EXACT COMMANDS TO UPGRADE YOUR INSTALLATION RIGHT NOW

    Since you’re clearly running in a conda (base) shell, use pip inside that environment:

    pip install --upgrade gallery-dl

    If gallery-dl was installed with pip3 instead:

    pip3 install --upgrade gallery-dl

    If it was installed under --user:

    pip install --upgrade --user gallery-dl

    After that:

    gallery-dl --version

    ✔ SUMMARY (QUICK)

    Tool Command to upgrade gallery-dl Why yours failed
    pip pip install --upgrade gallery-dl pip has no upgrade or update subcommand
    pipx pipx upgrade gallery-dl pipx wasn’t installed
    dnf sudo dnf upgrade python3-gallery-dl your repo doesn’t provide gallery-dl package

    If you want, I can walk you through:

    Just tell me.

    M: does the json format have a comment token

    G: Short answer:
    No. JSON does not support comments of any kind.
    There is no official comment token, not #, not //, not /* */.


    ✔ The long-but-still-straight answer

    1. The JSON specification (ECMA-404, RFC 8259) explicitly forbids comments.

    This is why:

    will all cause strict JSON parsers to throw an error.


    2. Some tools pretend to allow comments — but they are not JSON.

    Certain libraries (like json5, or some config loaders for VSCode, Node.js, etc.) allow:

    // comment
    /* comment */

    But that is JSON5, not JSON.

    gallery-dl uses strict JSON, not JSON5.


    ✔ So what can you do?

    Option A — fake a comment using a string key

    (You already did this, and it is the correct method.)

    Example:

    {
      "#": "This is a comment",
      "extractor": { ... }
    }

    or

    {
      "_comment": "Explanation goes here",
      "extractor": { ... }
    }

    Because JSON doesn’t forbid random keys, this is 100% valid.

    This is the standard hack used in config files for programs that require pure JSON.


    Option B — put comments in adjacent documentation, not inside JSON

    Not ideal, but safe.


    ✔ Summary

    Format Comments Allowed? Tokens
    JSON (what gallery-dl uses) ❌ No None
    JSON5 ✔ Yes //, /* */
    YAML ✔ Yes #
    TOML ✔ Yes #
    INI ✔ Yes ;, #

    If you'd like, I can help you restructure your config so “pseudo-comments” are grouped cleanly, readable, and impossible to conflict with real keys.

    M: it's working-ish - i just took out the filename stuff entirely to see how the rest was functioning - but there are two things. this has popped up some, not interrupting but i'd like to know if there's a fix: [formatter][error] Invalid format string '-XMP-dc:Subject<=${metadata[tags]|[]|join(',',_)}' (ValueError: Empty attribute in format string) [postprocessor][error] 'exec' initialization failed: ValueError: Empty attribute in format string

    ...and there's also a danbooru-type site that keeps doing this: AuthRequired: 'api-key' & 'user-id' needed to access the API ('Missing authentication.')

    G: Here is the clear, focused explanation of both issues — no overthinking, just what’s causing them and what your options actually are.


    1. The exec postprocessor formatting error

    You’re seeing:

    [formatter][error] Invalid format string '-XMP-dc:Subject<=${metadata[tags]|[]|join(',',_)}'
    (ValueError: Empty attribute in format string)
    [postprocessor][error] 'exec' initialization failed

    Why this happens

    Because gallery-dl’s formatter does NOT allow nested expressions inside ${...} when used in an exec command block.

    This part:

    ${metadata[tags]|[]|join(',',_)}

    is using advanced filters (join, fallback, array indexing).
    That syntax is valid in filenames or metadata templates, but NOT inside the exec-command formatter.

    Inside exec commands, gallery-dl tries to parse your string with Python’s str.format() rules → which cannot parse:

    Hence:

    ValueError: Empty attribute in format string

    In short:

    ExifTool tag embedding cannot use gallery-dl templating in this way.
    Exif metadata embedding can work — but not with that template syntax.

    Your options

    1. Remove the exec block entirely (simplest; everything else continues working).
    2. Keep the JSON metadata sidecar (you already generate it).
    3. If you really want ExifTool embedding, I can write a working version — but it must use:
      • an external script, OR
      • a simpler gallery-dl variable (e.g., ${tags} if the site exposes it directly)

    Right now:
    Your configuration is correct everywhere else; only the exec=EXIF block is invalid.


    2. The “AuthRequired: api-key & user-id needed” error

    You’re seeing something like:

    AuthRequired: 'api-key' & 'user-id' needed to access the API ('Missing authentication.')

    This is not an error in your config.
    This is the site itself.

    ✔ Why it happens

    Some Danbooru-type sites (e.g., Gelbooru, Moebooru forks, Philomena forks, Sankaku, etc.) require:

    to access:

    Without those credentials, the extractor switches to:

    ✔ How to fix it (IF you want authenticated access)

    Inside your gallery-dl config:

    {
      "extractor": {
        "danbooru": {
          "user-id": "YOUR_USER_ID",
          "api-key": "YOUR_API_KEY"
        }
      }
    }

    or for whichever site is giving the error:

    "gelbooru": { ... }
    "sankaku": { ... }
    "moebooru": { ... }
    "philomena": { ... }

    The exact field names differ per extractor.

    If you tell me the URL of the site causing the message, I can give the exact block needed.

    ✔ If you do not want to authenticate

    There is no fix — the site is intentionally blocking unauthenticated API usage.

    But gallery-dl will still download the images if the site allows non-API scraping.
    Some Danbooru forks do not allow this; they fully require login.


    ✔ Summary (short version)

    1. EXIF embedding error

    2. “api-key & user-id required”


    If you want:

    Just tell me which direction you prefer.

    M: kind of thought that last bit. ... is there any gallery-dl mode which causes it to function more directly like a browser to parse things and decide what to download that way, rather than using the api? and if not, is there a workaround - either an alternate but similar utility, or some form of simple wrapper? (don't give the code if you get that far - just confirm/deny the possibility.)

    G: Here is the direct, unambiguous answer to each part:


    1. Does gallery-dl have a mode to behave like a browser (HTML parsing only, no API)?

    Yes — but only for some sites.

    gallery-dl has two kinds of extractors:

    A) API extractors

    B) HTML/regex/DOM extractors


    2. Does gallery-dl have a switch to force HTML scraping instead of API?

    For Danbooru-type sites: absolutely not.
    The maintainers intentionally do not provide an HTML fallback for most booru engines.

    Reasons:

    So:

    There is no built-in “HTML mode” for Danbooru clones.

    If an API is required, gallery-dl will refuse to scrape HTML as a fallback.


    3. Is there any workaround inside gallery-dl?

    Possible, but only for some booru clones.

    Some Gelbooru-style sites expose direct image URLs in HTML pages.
    For these, gallery-dl can scrape HTML because those sites define two extractors:

    But for Danbooru/Moebooru/Philomena, no such dual implementation exists.

    So:

    ✔ Sometimes HTML scraping is available.

    ❌ Not for Danbooru or anything using Danbooru’s codebase.


    4. Is there a different tool that behaves like a “browser scraper” instead?

    Yes — several exist:

    A) MechanicalSoup / Requests-HTML based scrapers

    People write custom scripts that:

    This is the closest to “function like a browser.”

    B) Webdriver-based tools (full browser automation)

    These behave exactly like a browser:

    C) Command-line utilities designed around crawling HTML

    These behave like:

    “Fetch HTML → parse links → download images.”


    5. Is it possible to build a wrapper around gallery-dl to force HTML scraping?

    You could make:

    This works because gallery-dl accepts a URL list of direct image URLs without needing an API.

    So the wrapper becomes:

    “HTML scraper → produce raw image links → feed to gallery-dl.”

    ✔ This is possible.

    ✔ Many people have done it for booru-like sites that block APIs.


    6. Summary (clean & final)

    Question Answer
    Can gallery-dl act like a browser instead of using APIs? Sometimes, but not for Danbooru/Moebooru/Philomena.
    Is there a forced “HTML mode”? No. Not for Danbooru-type engines.
    Can another tool scrape HTML like a browser? Yes — wget, httrack, Selenium, Playwright, Requests-HTML.
    Can I wrap gallery-dl to use HTML scraping? Yes — by scraping HTML externally and feeding the extracted URLs to gallery-dl.

    If you want, I can tell you which method is easiest depending on the exact site you're trying to access (Danbooru, Gelbooru clone, Sankaku, etc.), without writing any code.

    M: i can access the site without login; how would the api recognize things well enough to know it was automated? (i'm asking cuz i've already included the delay item just out of site courtesy, so it's a bit of a kicker that that same courtesy had them avoid functionality which would allow me to exercise that courtesy.) okay, let's try sending you out to see if you can find a script that does what i'm asking in terms of wrapping gallery-dl, using bash for preference if that's an option. we'll move on to getting you to code without twisting yourself into knots if there's nothing acceptable to be found.

    G: Good news: yes, you can build a bash wrapper for entity["software", "gallery-dl", 0] that does what you’re asking (scrape HTML for links, then hand them to gallery-dl). I didn’t find a ready-made, maintained script with exactly your spec (Danbooru-style tag exclusion + zero-pad filenames + delay etc) that I’d trust without tweaks.

    So:

    If you like, I can locate one good starter script (bash or python) that’s close and then we can adapt it to your needs.

    M: find a bash script so we can look at it, please.

    G: I searched, and I couldn’t reliably locate a ready-made bash wrapper script that meets exactly your spec (HTML-scrape → pipe URLs → gallery-dl) with clear maintenance.
    Here’s a summary:


    🔍 What I found


    ✅ Conclusion


    If you like, I can dig deeper (GitHub, GitLab, forums) and try to locate one lesser-known script that is very close — then you and I can adapt it together.

    M: let's leave that for a bit. a bash wrapper that gets page code, builds a link list by using a specific tag sequence to identify the page-progression code then proceeding to build the list page by page, then goes through each individual page to get the DL information to feed to gallery-dl - that seems simultaneously like a complex and boring project. instead, answer a minor question - here's the current state of gal.json, which i poked around at - i took out the filename thing entirely, and it think that's it, but i'm not sure. anyway, if i wanted to make it so that files all got popped out beginning with the information from the... i don't know the accurate term, but the test output gives json files containing the fields 'no' and 'filename' that i want to use for the file titles. can i get some syntax, please?

    G: Here is the clean, direct answer about how to build filenames from the two fields you mentioned:

    (and optionally still append the extension)

    All of this is based on the config you uploaded: fileciteturn1file0

    No overthinking — just exact syntax and what works in gallery-dl's formatter.


    ✔️ 1. If you want the filename to start with the post number (no)

    (typical for 4chan and many booru clones)

    Use:

    "filename": "{no}.{extension}"

    Zero-padding:

    "filename": "{no:0>9}.{extension}"

    This means:


    ✔️ 2. If you want: postnumber + original-filename

    (using the site’s provided filename field)

    Syntax:

    "filename": "{no:0>9}_{filename}.{extension}"

    Examples:


    ✔️ 3. If the site sometimes lacks no or filename,

    use fallbacks:

    "filename": "{no|filename|id|index:0>9}.{extension}"

    Interpreter logic:

    1. Try no
    2. If missing, try filename
    3. If missing, try id
    4. If missing, use gallery-dl’s internal index counter
    5. Apply zero-padding
    6. Append the actual extension

    This guarantees you never get None.ext.


    ✔️ 4. Where to put this in your config

    In your uploaded config, insert this right under "archive":

    "filename": "{no:0>9}_{filename}.{extension}",

    Or whichever variant you choose.

    Example inserted:

    "archive": "~/Downloads/gallery-dl/archive.sqlite3",
    "filename": "{no:0>9}_{filename}.{extension}",
    "#": "Default filename: site/post-or-thread/NNN.ext with zero padding",

    That’s it.


    ✔️ 5. Confirm the field names gallery-dl exposes

    If unsure, run:

    gallery-dl -K "URL"

    This prints all available metadata fields for that extractor — including no, filename, id, etc.
    You already verified no and filename appear in the JSON sidecar, so using them is valid.


    ✔️ 6. One final detail (important but simple)

    filename in gallery-dl does NOT include the extension.
    So your pattern must always append {extension} yourself:

    Correct:

    {filename}.{extension}

    Incorrect:

    {filename}

    (because the file would be saved without .jpg, .png, .webm, etc.)


    ✔️ Summary — usable syntax blocks

    A — Post number only

    "filename": "{no:0>9}.{extension}"

    B — Post number + original filename

    "filename": "{no:0>9}_{filename}.{extension}"

    C — Complete fallback-safe version

    "filename": "{no|filename|id|index:0>9}.{extension}"

    If you tell me which one of these you want as your final default, I’ll drop it cleanly into your current config without touching anything else.

    M: do you have any idea when the danbooru-style sites started being jealous with their api? because i know it's been a bit since i used gallery-dl, but i didn't think it was that long ago... also, please look around online and see if there's any indication that the api stuff has been getting used to ban users. i don't mind the idea of signing in if it becomes a good idea for a project or whatever - but i don't want to piss off the operators, i'd rather work on workarounds.

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: reasoning_recap; content_type=reasoning_recap]

    G: Short version:

    Details:


    1. “When did they get jealous with their API?”

    Danbooru proper

    So:
    You probably last used gallery-dl in the era where:

    That shifted noticeably in the early–mid 2020s, and particularly hard in the last couple of years.

    Danbooru-style / booru clones (e.g. rule34.xxx)

    This is even more “per-site & recent”:

    So your impression is right:
    It was easier recently, and some of these sites have only just flipped the “API auth mandatory” bit in the last year or so.


    2. “If I can browse without login, how does the API know it’s automated?”

    Two separate pieces:

    Your 2-second sleep is good manners, but they’re reacting to the global behavior of every bot and scraper, not just you.

    The shift to “you must authenticate to use the API” is basically them saying:

    “If you’re going to automate, do it where we can rate-limit and attribute it cleanly.”


    3. “Are APIs being used to ban users?”

    What I can see:

    I haven’t found reputable reports of:

    What does show up:

    So the reality looks like:


    4. If you want to be “polite” and avoid pissing off operators

    Given your goals:

    If you truly don’t want to touch their API at all, HTML-scraping workarounds are an option, but from the operator’s point of view that’s usually less polite, not more — HTML is heavier and harder to control than an authenticated API with rate limits.


    Bottom line:

    M: Okay, it must have been around August or September that I last used it, because it wasn't around 2013. And I go from site to site, so... For all I know, this new one... I'll go back and check and see if the old one still works. It could just be that it's the new one, anyway. Okay, let's expand thoughts, though. If I just want to sledgehammer my way through the problem, I really like GalleryDL in terms of its flexibility and capabilities. If I did want to take a sledgehammer to the problem by shifting utilities, you gave me a list before, given my setup and preference for command lines and scriptability. What would be the best choices for me as a second tool?

    G: Here’s the clean, no-nonsense answer, tuned specifically to your preferences:

    I’ll break it into three categories, and then give you the “if you only want one alternate tool, pick THIS” recommendation.


    ✔ CATEGORY 1 — “Browser-like HTML scrapers” (Direct, simple, scriptable)

    These tools pull HTML and extract links like a browser would, with zero API dependency.
    They’re the closest match to “I want something that does its own parsing and doesn’t ask permission.”

    1. wget (recursive mode)

    Strength: ancient, solid, scriptable, handles pagination, cookies, user agents.
    Weakness: crude; can easily miss JS-generated content.

    Used like:

    wget --recursive --accept jpg,png,gif,webm --no-parent URL

    Great for sites where images are just <img src="..."> or clickable links in HTML.


    2. httrack

    Strength: full website mirroring; understands relative paths, link rewriting, pagination.
    Weakness: heavyweight; can overshoot and download more than you want.

    httrack 'https://example.com/gallery/' -O output -%v

    Good when the site is purely HTML with straightforward link relationships.


    3. hakrawler

    A newer, faster HTML link crawler designed for pentesting but useful for gathering links.
    Strong CLI, pipes well into gallery-dl.

    hakrawler -url https://site/page | grep -E '\.(jpg|png|gif|webm)$' | gallery-dl -i -

    Strength: fast, good for collecting URLs without API.
    Weakness: no tag logic, no metadata.


    ✔ CATEGORY 2 — “Scriptable HTML parsing” (flexible + programmable)

    These are for when HTML needs actual parsing logic — like extracting a specific tag, finding a class name, or stepping pagination links.

    4. pup (HTML selector tool)

    Think: jq, but for HTML.
    This is the closest to “browser-ish parsing” while staying fully CLI-native.

    Example:

    curl -s URL | pup 'img[src] attr{src}'

    You can then pipe into gallery-dl:

    curl -s URL | pup 'img[src] attr{src}' | gallery-dl -i -

    Strength: extremely scriptable, stays in bash, works exactly how you think.
    Weakness: won’t handle JS-heavy sites.


    5. htmlq (another jq-style HTML parser)

    Lighter than pup.
    Uses CSS selectors:

    htmlq -a src img < page.html

    6. mechanicalsoup (Python)

    If you ever decide to go slightly beyond bash, MechanicalSoup behaves roughly like a minimalist, headless browser without all the Selenium overhead.

    Strength: easily follows forms, pagination, cookies.
    Weakness: not strictly bash, but extremely lightweight.


    ✔ CATEGORY 3 — “Real browser automation” (nuclear option)

    If a site uses JavaScript or blocks HTML-only scraping, these tools let you fetch DOM-rendered content exactly like a browser would.

    You said you prefer CLI/scriptability, so I’m excluding GUI-first tools and going directly to headless/controllable ones:

    7. Playwright (headless mode)

    The most stable option in 2025.
    You can write scripts that:

    But it requires a Python or Node wrapper; not pure bash.


    8. Selenium headless

    Similar, but heavier and more finicky.


    ✔ Now, the real answer you want:

    ⭐ BEST PRACTICAL SECOND TOOL (Sledgehammer + Your Preferences)

    Here’s why:

    ✔ Stays in bash

    (no Python, no browser engines, no headless Chrome)

    ✔ Extremely scriptable

    You can cut-and-paste it into tiny one-liners or larger scripts.

    ✔ Works with pagination logic

    You can parse out “next page” links and loop over them.

    Meaning: gallery-dl still handles:

    Your bash wrapper just supplies the image URLs.

    ✔ Stable and lightweight

    Pup is tiny, maintained, and does not rely on APIs.


    ✔ If you want the nuclear “guaranteed to work” option:

    Playwright (CLI + Python wrapper)

    This is for when:

    It is the sledgehammer that always works — but heavier to install.


    ✔ If you want the simplest HTML-only hammer:

    Example:

    hakrawler -url https://example.com | grep -E '\.(jpg|png|gif|webm)$' | gallery-dl -i -

    ✔ Final distilled recommendation (based on your personality + workflow)

    You already have gallery-dl set up nicely. You want another tool to “smash through API restrictions” by feeding gallery-dl direct URLs.
    You prefer bash and command-line scriptability.

    Therefore:

    👉 Pick: curl + pup as your secondary tool.

    It gives you:

    And if the site is fussy or JS-heavy:

    👉 Your fallback sledgehammer is: Playwright headless.


    If you want, I can design:

    Just tell me which direction you’d like.

    M: I might try each of your three, but here's the question. Again, Linux setup Fedora. That second part is important because that seems to be kind of the RC Cola to other Linux builds, Coke and Pepsi. Ease of installation, uninstallation, and program size. I don't mind trying them out and then letting them sit until the next time I have to wipe my entire system as long as it isn't going to create a situation where every update involves five different forks or half a gig of file size accidentally.

    G: Got it. You want the simplest, smallest, cleanest-install tools that won’t:

    And you’re on Fedora, which is indeed the RC Cola of the Linux world: absolutely solid, but many packages default to “developer-friendly over lightweight.”

    So here’s the straight, practical breakdown:


    ✔️ Tier 1 — Lightweight, safe to install, safe to forget

    These add no heavy dependencies, uninstall cleanly, and won’t mutate over time.

    1. pup

    Fedora often doesn’t ship pup in official repos. So the safest method is:

    cd ~/bin
    curl -LO https://github.com/ericchiang/pup/releases/download/v0.4.0/pup-linux-amd64
    mv pup-linux-amd64 pup
    chmod +x pup

    That’s it.
    No root required, no dependencies.
    If you don’t want it anymore: delete ~/bin/pup.

    Verdict:
    This is the cleanest, safest choice of all the tools.


    2. hakrawler

    Installation same style as pup:

    cd ~/bin
    curl -LO https://github.com/hakluke/hakrawler/releases/latest/download/hakrawler_linux_amd64
    mv hakrawler_linux_amd64 hakrawler
    chmod +x hakrawler

    Verdict:
    Still extremely lightweight—safe to install and forget.
    Perfect for “grab every link beginning with X.”


    3. wget (recursive)

    Fedora already has it installed.

    Verdict:
    Already there; just use it.


    ✔️ Tier 2 — Slightly heavier but still OK

    These install from Fedora repos and add minimal dependencies.

    4. htmlq

    Verdict:
    Still lightweight; Fedora packaging for Rust is very clean.


    ✔️ Tier 3 — Heavy (avoid unless you need JS rendering)

    5. Playwright

    This is the real sledgehammer.
    Installing Playwright on Fedora pulls Chromium, Firefox, and WebKit binaries, and the Python wrapper has a large dependency tree.

    On Fedora, the command:

    pip install playwright
    playwright install

    Downloads multiple full browsers, which you probably don’t want.

    Verdict:
    Use only if a site is fully JS-rendered and nothing else works.


    ✔️ Final Recommendation (Based on Your Constraints)

    You want:

    So here’s the clear verdict:

    Best Secondary Tools for You

    3. htmlq (clean, jq-like selector tool)

    4. wget (already installed anyway)

    These four tools are:

    Avoid (unless absolutely necessary)

    Playwright, Selenium, anything headless-browser-based

    (due to size, complexity, and messy uninstall on Fedora)


    ✔️ Which should you try first?

    If you want to explore them in order of “least risk → most power”:

    1. pup
    2. hakrawler
    3. htmlq
    4. wget recursive

    All are under 10MB and won't pollute your system.


    If you like, I can walk you through installing all four in a dedicated folder, so they never touch system paths and can be removed with one rm -r.

    M: back trying to gallery-dl to do what i want, on a new ubuntu install. can you give me a local json that does what i've been asking you to do here? or are the tokens old enough that we need to do a refresher?

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: reasoning_recap; content_type=reasoning_recap]

    G: Yes — I have enough to give you a fresh local config without a full refresher.

    This should match the stable goals you’ve been building toward:

    gallery-dl supports JSON config files, sleep, archive, filename formatting, and metadata postprocessors; it also supports YAML/TOML alternatives, but this version is plain JSON. citeturn129075search0turn129075search3turn129075search9

    Use this as a local gal.json:

    {
      "extractor": {
        "base-directory": "~/Downloads/gallery-dl",
    
        "path-restrict": {
          " ": "_",
          "/": "_",
          "\\": "_",
          "|": "_",
          ":": "_",
          "*": "_",
          "?": "_",
          "\"": "_",
          "<": "_",
          ">": "_"
        },
        "path-remove": "\u0000-\u001f\u007f",
        "path-replace": "_",
    
        "sleep": 2.0,
        "sleep-request": 0.0,
        "sleep-extractor": 0.0,
    
        "archive": "~/Downloads/gallery-dl/archive.sqlite3",
    
        "directory": ["{category}", "{title|id}"],
        "filename": "{no:0>9}_{filename}.{extension}"
      },
    
      "format": {
        "file": {
          "extensions": ["webm", "mp4"]
        }
      },
    
      "postprocessors": [
        { "name": "metadata", "mode": "json", "event": "after" },
        { "name": "metadata", "mode": "tags", "event": "after" }
      ]
    }

    A couple of important notes:

    Your previous exec / exiftool block is the part that was blowing up. The formatter error came from that command template, so I left it out here. The built-in metadata postprocessor is documented and is the safe part to keep. citeturn129075search0turn129075search6

    If you want the filename to be more fault-tolerant across mixed sites, use this instead of the filename line above:

    "filename": "{no|filename|id|index}.{extension}"

    or, if you want the leading-zero rule only when no exists but still avoid None.ext, use:

    "filename": "{no:0>9}_{filename|id|index}.{extension}"

    The reason I’m splitting those is that gallery-dl’s formatter supports fallback chains, but formatting mixed text-and-number fallbacks can get ugly if you try to force zero-padding onto a non-numeric fallback. The examples and discussions around filename formatting and duplicate-file behavior back up being conservative here. citeturn129075search2turn129075search11

    Run it locally with:

    gallery-dl -c ./gal.json -i list.txt

    And if you want to check the exact available fields for a site before locking the filename pattern in, use:

    gallery-dl -K "URL"

    That is still the quickest way to verify whether a given extractor exposes no, filename, id, or something else. citeturn129075search9turn129075search17

    The one thing I’m not fully certain about without seeing your current target site is whether no is always present there. If it isn’t, I’d use the second, safer filename line instead of the first.

    M: this is for just video files, yes? what would i need to either add or remove to download all media?

    G: [Non-text content: thoughts; content_type=thoughts]

    G: [Non-text content: reasoning_recap; content_type=reasoning_recap]

    G: Yes. In that config, this part is what limits it to video files:

    "format": {
      "file": {
        "extensions": ["webm", "mp4"]
      }
    }

    If you want all media, remove that entire block.

    So change from:

    {
      "extractor": {
        ...
      },
    
      "format": {
        "file": {
          "extensions": ["webm", "mp4"]
        }
      },
    
      "postprocessors": [
        ...
      ]
    }

    to:

    {
      "extractor": {
        ...
      },
    
      "postprocessors": [
        ...
      ]
    }

    That will let gallery-dl download whatever media the extractor normally exposes: images, gifs, videos, whatever the site provides.

    If later you want a mixed but selective set instead, you can use something like:

    "extensions": ["jpg", "jpeg", "png", "gif", "webm", "mp4"]

    But for everything, the answer is just: remove the format.file.extensions block entirely.

    M: thought so but had to check. ... i'll prolly be back at some point. but thank you for getting it together.

    G: Fair call—that was the only piece gating it.

    What you’ve got now is basically a clean, stable baseline config:

    That’s about as “set it and forget it” as gallery-dl gets without going into site-specific tuning.

    If you come back later, the next likely pressure points will be:

    But for now, you’re in a good place.

    M: oh, yes - what is the config specifying flag? -c?

    G: Yes — it’s:

    -c

    ✔ Basic usage

    gallery-dl -c gal.json URL

    or with your list:

    gallery-dl -c gal.json -i list.txt

    ✔ Full form (same thing)

    --config

    Example:

    gallery-dl --config gal.json -i list.txt

    ✔ Important behavior (worth knowing)


    That’s it — no hidden quirks beyond that.