Skip to main content

Link Resolution

link_resolution defines how the engine follows and unpacks redirect/obfuscated links before downloading. It's a sequential step system for resolving a single URL through a chain of fetches and extractions.

For complex multi-step navigation (branching, loops, conditional logic), use Navigation Paths instead.


Structure

link_resolution:
enabled: true
resolvers:
resolver_name:
source_field: link
steps:
- action: fetch
...
- action: extract
...

Steps use action: as a tag — this is different from the - fetch: {...} format used in navigation paths.


Steps

fetch

- action: fetch
follow_redirects: true
headers:
Referer: "https://example.com"
timeout_ms: 10000

extract

- action: extract
method: regex
pattern: "window\\.location = ['\"]([^'\"]+)['\"]"
group: 1
- action: extract
method: selector
selector: "a#download-link"
attribute: href
- action: extract
method: json_path
pattern: "data.download_url"
FieldNotes
methodregex, selector, json_path, xpath, text
patternFor regex / json_path
groupRegex capture group (default: 0)
selectorCSS selector for selector method
attributeHTML attribute to extract
fallbackValue to use if extraction fails

transform

- action: transform
transformations:
- type: trim
- type: url_normalize
base_url: https://example.com

wait

- action: wait
duration_ms: 2000

Examples

JavaScript redirect page

link_resolution:
resolvers:
redirect_page:
source_field: link
steps:
- action: fetch
follow_redirects: true
- action: extract
method: regex
pattern: "location\\.href=['\"]([^'\"]+)['\"]"
group: 1

HTML page with a download button

link_resolution:
resolvers:
download_page:
source_field: link
steps:
- action: fetch
follow_redirects: true
- action: extract
method: selector
selector: "a.download-button"
attribute: href
- action: transform
transformations:
- type: url_normalize
base_url: https://example.com

Build URL from extracted ID

link_resolution:
resolvers:
id_to_url:
source_field: link
steps:
- action: extract
method: regex
pattern: "/file/([A-Za-z0-9]+)"
group: 1
- action: transform
transformations:
- type: template
template: "https://cdn.example.com/download/{value}"

Use caseTool
Follow a redirect or extract a URL from a pagelink_resolution
Multiple steps, branching, loops, conditionspaths
Hosters page with many mirrorspaths
Per-host programmatic downloadhosts.resolver