22 Commits

Author SHA1 Message Date
Andy
bc26bf3046 feat: update changelog for version 1.4.7 2025-09-25 06:29:46 +00:00
Andy
35efdbff6d feat: add curl_cffi session support with browser impersonation
Add new session utility with curl_cffi support for anti-bot protection
Update all manifest parsers (DASH, HLS, ISM, M3U8) to accept curl_cffi sessions
Add browser impersonation support (Chrome, Firefox, Safari)
Fix cookie handling compatibility between requests and curl_cffi
Suppress HTTPS proxy warnings for better UX
Maintain full backward compatibility with requests.Session
2025-09-25 06:27:14 +00:00
Andy
63b7a49c1a feat: Add decrypt_labs_api_key to Config initialization and change duplicate track log level to debug 2025-09-25 06:22:50 +00:00
Andy
98ecf6f876 feat: Add download retry count option to download function 2025-09-23 01:32:00 +00:00
Andy
5df6914536 feat: Add options for required subtitles and best available quality in download command 2025-09-23 01:28:55 +00:00
Andy
c1df074965 Change new dynamic CDM selection text to be in Debug only 2025-09-14 04:25:57 +00:00
Andy
da60a396dd Fix: Prevent KeyError when reusing remote CDMs in dynamic selection
Creates a copy of the CDM dictionary before modification to prevent the original configuration from being mutated, allowing the same CDM to be selected multiple times within a session without errors.
2025-09-14 01:14:01 +00:00
Andy
a99a391395 chore: bump version to 1.4.6 and update changelog 2025-09-13 04:01:45 +00:00
Andy
ed32939d83 feat: Add quality-based CDM selection for dynamic CDM switching
Implements dynamic CDM selection based on video track resolution to optimize
CDM usage. Automatically selects appropriate security level (L3/SL2K for ≤1080p, L1/SL3K for >1080p) based on content requirements.

Key Features:
- Quality-based CDM configuration with threshold operators (>=, >, <=, <)
- Pre-selection based on highest quality across all video tracks
- Maintains backward compatibility with existing CDM configurations
- Single CDM per session to avoid inefficient switching
2025-09-13 03:59:13 +00:00
Andy
4006593a8a Fix: Implement lazy DRM loading for multi-track key retrieval
- Add deferred DRM loading to M3U8 parser to mark tracks for later processing
- Optimize prepare_drm to load DRM just-in-time during download process
2025-09-12 06:38:14 +00:00
Andy
307be4549b Fix vault caching count and NoneType iteration issues
- Fix 'NoneType' object is not iterable error in decrypt_labs_remote_cdm
- Fix vault count display showing 0/3 instead of actual successful vault count
2025-09-10 06:33:46 +00:00
Andy
a82828768d feat: automatic audio language metadata for embedded audio tracks
- Add intelligent embedded audio language detection at mux stage
- Automatically set audio language metadata when no separate audio tracks exist
- Respect user flags (-V, --no-audio) to avoid unnecessary processing
- Smart video track selection based on title language with fallbacks
- Improved default track selection to prioritize title language matches
- Enhanced FFmpeg repackaging with audio stream metadata injection
- Works automatically for all services without service-specific code
2025-09-10 00:57:14 +00:00
Andy
d18a5de0d0 fix: Improve import ordering and code formatting
- Reorder imports in decrypt_labs_remote_cdm.py for better organization
- Clean up trailing whitespace in SQLite.py
2025-09-10 00:53:52 +00:00
Andy
04b540b363 fix: Resolve service name transmission and vault case sensitivity issues
Fixed DecryptLabsRemoteCDM sending 'generic' instead of proper service names and added case-insensitive vault lookups for SQLite/MySQL vaults. Also added local vault integration to DecryptLabsRemoteCDM
2025-09-09 18:53:11 +00:00
Andy
6137146705 chore: bump version to 1.4.5 and update changelog
- Update version from 1.4.4 to 1.4.5 in core/__init__.py
- Add comprehensive changelog entry for v1.4.5 with all changes since 1.4.4
- Include enhanced CDM support, caching improvements, and bug fixes
2025-09-09 03:53:42 +00:00
Andy
859d09693c feat(cdm): Update User-Agent to use dynamic version
- Replace hardcoded version "1.0" with dynamic version import in DecryptLabsRemoteCDM User-Agent header.
2025-09-09 03:49:01 +00:00
Andy
5f022635cb feat(cdm): Optimize get_cached_keys_if_exists for L1/L2 devices
- Always send get_cached_keys_if_exists=True for L1/L2 devices to leverage
- the API's automatic caching optimization. This reduces unnecessary license
- requests by prioritizing cached keys for these security levels.
2025-09-06 22:10:35 +00:00
Andy
ad66502c0c feat(cdm): Add fallback to Widevine common cert for L1 devices
- Use default Widevine common privacy certificate when no service certificate is provided for L1 devices
- Add get_widevine_service_certificate method to EXAMPLE service for config-based certificates
- Improve certificate handling with more descriptive return messages
2025-09-06 20:30:11 +00:00
Andy
e462f07b7a Merge branch 'main' of https://github.com/unshackle-dl/unshackle 2025-09-06 19:39:39 +00:00
Andy
83b600e999 fix(cdm): Clean up session data when retrieving cached keys
Remove decrypt_labs_session_id and challenge from session when cached keys exist but there are missing kids, ensuring clean state for subsequent requests.
2025-09-06 19:38:54 +00:00
Andy
ea8a7b00c9 fix(cdm): Clean up session data when retrieving cached keys
Remove decrypt_labs_session_id and challenge from session when cached keys exist but there are missing kids, ensuring clean state for subsequent requests.
2025-09-06 18:52:20 +00:00
Andy
16ee4175a4 feat(dl): Truncate PSSH string for display in non-debug mode
* Added `_truncate_pssh_for_display` method to limit the width of PSSH strings shown in the console.
* Ensures better readability of DRM information by truncating long strings.
2025-09-05 02:15:10 +00:00
21 changed files with 902 additions and 274 deletions

View File

@@ -5,6 +5,120 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.4.7] - 2025-09-25
### Added
- **curl_cffi Session Support**: Enhanced anti-bot protection with browser impersonation
- Added new session utility with curl_cffi support for bypassing anti-bot measures
- Browser impersonation support for Chrome, Firefox, and Safari user agents
- Full backward compatibility with requests.Session maintained
- Suppressed HTTPS proxy warnings for improved user experience
- **Download Retry Functionality**: Configurable retry mechanism for failed downloads
- Added retry count option to download function for improved reliability
- **Subtitle Requirements Options**: Enhanced subtitle download control
- Added options for required subtitles in download command
- Better control over subtitle track selection and requirements
- **Quality Selection Enhancement**: Improved quality selection options
- Added best available quality option in download command for optimal track selection
- **DecryptLabs API Integration**: Enhanced remote CDM configuration
- Added decrypt_labs_api_key to Config initialization for better API integration
### Changed
- **Manifest Parser Updates**: Enhanced compatibility across all parsers
- Updated DASH, HLS, ISM, and M3U8 parsers to accept curl_cffi sessions
- Improved cookie handling compatibility between requests and curl_cffi
- **Logging Improvements**: Reduced log verbosity for better user experience
- Changed duplicate track log level to debug to reduce console noise
- Dynamic CDM selection messages moved to debug-only output
### Fixed
- **Remote CDM Reuse**: Fixed KeyError in dynamic CDM selection
- Prevents KeyError when reusing remote CDMs in dynamic selection process
- Creates copy of CDM dictionary before modification to prevent configuration mutation
- Allows same CDM to be selected multiple times within session without errors
## [1.4.6] - 2025-09-13
### Added
- **Quality-Based CDM Selection**: Dynamic CDM selection based on video resolution
- Automatically selects appropriate CDM (L3/L1) based on video track quality
- Supports quality thresholds in configuration (>=, >, <=, <, exact match)
- Pre-selects optimal CDM based on highest quality across all video tracks
- Maintains backward compatibility with existing CDM configurations
- **Automatic Audio Language Metadata**: Intelligent embedded audio language detection
- Automatically sets audio language metadata when no separate audio tracks exist
- Smart video track selection based on title language with fallbacks
- Enhanced FFmpeg repackaging with audio stream metadata injection
- **Lazy DRM Loading**: Deferred DRM loading for multi-track key retrieval optimization
- Add deferred DRM loading to M3U8 parser to mark tracks for later processing
- Just-in-time DRM loading during download process for better performance
### Changed
- **Enhanced CDM Management**: Improved CDM switching logic for multi-quality downloads
- CDM selection now based on highest quality track to avoid inefficient switching
- Quality-based selection only within same DRM type (Widevine-to-Widevine, PlayReady-to-PlayReady)
- Single CDM used per session for better performance and reliability
### Fixed
- **Vault Caching Issues**: Fixed vault count display and NoneType iteration errors
- Fix 'NoneType' object is not iterable error in DecryptLabsRemoteCDM
- Fix vault count display showing 0/3 instead of actual successful vault count
- **Service Name Transmission**: Resolved DecryptLabsRemoteCDM service name issues
- Fixed DecryptLabsRemoteCDM sending 'generic' instead of proper service names
- Added case-insensitive vault lookups for SQLite/MySQL vaults
- Added local vault integration to DecryptLabsRemoteCDM
- **Import Organization**: Improved import ordering and code formatting
- Reorder imports in decrypt_labs_remote_cdm.py for better organization
- Clean up trailing whitespace in vault files
### Configuration
- **New CDM Configuration Format**: Extended `cdm:` section supports quality-based selection
```yaml
cdm:
SERVICE_NAME:
"<=1080": l3_cdm_name
">1080": l1_cdm_name
default: l3_cdm_name
```
## [1.4.5] - 2025-09-09
### Added
- **Enhanced CDM Key Caching**: Improved key caching and session management for L1/L2 devices
- Optimized `get_cached_keys_if_exists` functionality for better performance with L1/L2 devices
- Enhanced cached key retrieval logic with improved session handling
- **Widevine Common Certificate Fallback**: Added fallback to Widevine common certificate for L1 devices
- Improved compatibility for L1 devices when service certificates are unavailable
- **Enhanced Vault Loading**: Improved vault loading and key copying logic
- Better error handling and key management in vault operations
- **PSSH Display Optimization**: Truncated PSSH string display in non-debug mode for cleaner output
- **CDM Error Messaging**: Added error messages for missing service certificates in CDM sessions
### Changed
- **Dynamic Version Headers**: Updated User-Agent headers to use dynamic version strings
- DecryptLabsRemoteCDM now uses dynamic version import instead of hardcoded version
- **Intelligent CDM Caching**: Implemented intelligent caching system for CDM license requests
- Enhanced caching logic reduces redundant license requests and improves performance
- **Enhanced Tag Handling**: Improved tag handling for TV shows and movies from Simkl data
- Better metadata processing and formatting for improved media tagging
### Fixed
- **CDM Session Management**: Clean up session data when retrieving cached keys
- Remove decrypt_labs_session_id and challenge from session when cached keys exist but there are missing kids
- Ensures clean state for subsequent requests and prevents session conflicts
- **Tag Formatting**: Fixed formatting issues in tag processing
- **Import Order**: Fixed import order issues in tags module
## [1.4.4] - 2025-09-02
### Added

View File

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "unshackle"
version = "1.4.4"
version = "1.4.6"
description = "Modular Movie, TV, and Music Archival Software."
authors = [{ name = "unshackle team" }]
requires-python = ">=3.10,<3.13"

View File

@@ -66,6 +66,18 @@ from unshackle.core.vaults import Vaults
class dl:
@staticmethod
def _truncate_pssh_for_display(pssh_string: str, drm_type: str) -> str:
"""Truncate PSSH string for display when not in debug mode."""
if logging.root.level == logging.DEBUG or not pssh_string:
return pssh_string
max_width = console.width - len(drm_type) - 12
if len(pssh_string) <= max_width:
return pssh_string
return pssh_string[: max_width - 3] + "..."
@click.command(
short_help="Download, Decrypt, and Mux tracks for titles from a Service.",
cls=Services,
@@ -161,6 +173,12 @@ class dl:
help="Language wanted for Audio, overrides -l/--lang for audio tracks.",
)
@click.option("-sl", "--s-lang", type=LANGUAGE_RANGE, default=["all"], help="Language wanted for Subtitles.")
@click.option(
"--require-subs",
type=LANGUAGE_RANGE,
default=[],
help="Required subtitle languages. Downloads all subtitles only if these languages exist. Cannot be used with --s-lang.",
)
@click.option("-fs", "--forced-subs", is_flag=True, default=False, help="Include forced subtitle tracks.")
@click.option(
"--proxy",
@@ -251,6 +269,13 @@ class dl:
@click.option(
"--reset-cache", "reset_cache", is_flag=True, default=False, help="Clear title cache before fetching."
)
@click.option(
"--best-available",
"best_available",
is_flag=True,
default=False,
help="Continue with best available quality if requested resolutions are not available.",
)
@click.pass_context
def cli(ctx: click.Context, **kwargs: Any) -> dl:
return dl(ctx, **kwargs)
@@ -310,6 +335,16 @@ class dl:
vault_copy = vault.copy()
del vault_copy["type"]
if vault_type.lower() == "api" and "decrypt_labs" in vault_name.lower():
if "token" not in vault_copy or not vault_copy["token"]:
if config.decrypt_labs_api_key:
vault_copy["token"] = config.decrypt_labs_api_key
else:
self.log.warning(
f"No token provided for DecryptLabs vault '{vault_name}' and no global "
"decrypt_labs_api_key configured"
)
if vault_type.lower() == "sqlite":
try:
self.vaults.load_critical(vault_type, **vault_copy)
@@ -430,6 +465,7 @@ class dl:
v_lang: list[str],
a_lang: list[str],
s_lang: list[str],
require_subs: list[str],
forced_subs: bool,
sub_format: Optional[Subtitle.Codec],
video_only: bool,
@@ -450,6 +486,7 @@ class dl:
no_source: bool,
workers: Optional[int],
downloads: int,
best_available: bool,
*_: Any,
**__: Any,
) -> None:
@@ -457,6 +494,10 @@ class dl:
self.search_source = None
start_time = time.time()
if require_subs and s_lang != ["all"]:
self.log.error("--require-subs and --s-lang cannot be used together")
sys.exit(1)
# Check if dovi_tool is available when hybrid mode is requested
if any(r == Video.Range.HYBRID for r in range_):
from unshackle.core.binaries import DoviTool
@@ -691,8 +732,14 @@ class dl:
res_list = ", ".join([f"{x}p" for x in missing_resolutions[:-1]]) + " or "
res_list = f"{res_list}{missing_resolutions[-1]}p"
plural = "s" if len(missing_resolutions) > 1 else ""
self.log.error(f"There's no {res_list} Video Track{plural}...")
sys.exit(1)
if best_available:
self.log.warning(
f"There's no {res_list} Video Track{plural}, continuing with available qualities..."
)
else:
self.log.error(f"There's no {res_list} Video Track{plural}...")
sys.exit(1)
# choose best track by range and quality
if any(r == Video.Range.HYBRID for r in range_):
@@ -728,7 +775,21 @@ class dl:
title.tracks.videos = selected_videos
# filter subtitle tracks
if s_lang and "all" not in s_lang:
if require_subs:
missing_langs = [
lang
for lang in require_subs
if not any(is_close_match(lang, [sub.language]) for sub in title.tracks.subtitles)
]
if missing_langs:
self.log.error(f"Required subtitle language(s) not found: {', '.join(missing_langs)}")
sys.exit(1)
self.log.info(
f"Required languages found ({', '.join(require_subs)}), downloading all available subtitles"
)
elif s_lang and "all" not in s_lang:
missing_langs = [
lang_
for lang_ in s_lang
@@ -850,9 +911,40 @@ class dl:
selected_tracks, tracks_progress_callables = title.tracks.tree(add_progress=True)
for track in title.tracks:
if hasattr(track, "needs_drm_loading") and track.needs_drm_loading:
track.load_drm_if_needed(service)
download_table = Table.grid()
download_table.add_row(selected_tracks)
video_tracks = title.tracks.videos
if video_tracks:
highest_quality = max((track.height for track in video_tracks if track.height), default=0)
if highest_quality > 0:
if isinstance(self.cdm, (WidevineCdm, DecryptLabsRemoteCDM)) and not (
isinstance(self.cdm, DecryptLabsRemoteCDM) and self.cdm.is_playready
):
quality_based_cdm = self.get_cdm(
self.service, self.profile, drm="widevine", quality=highest_quality
)
if quality_based_cdm and quality_based_cdm != self.cdm:
self.log.debug(
f"Pre-selecting Widevine CDM based on highest quality {highest_quality}p across all video tracks"
)
self.cdm = quality_based_cdm
elif isinstance(self.cdm, (PlayReadyCdm, DecryptLabsRemoteCDM)) and (
isinstance(self.cdm, DecryptLabsRemoteCDM) and self.cdm.is_playready
):
quality_based_cdm = self.get_cdm(
self.service, self.profile, drm="playready", quality=highest_quality
)
if quality_based_cdm and quality_based_cdm != self.cdm:
self.log.debug(
f"Pre-selecting PlayReady CDM based on highest quality {highest_quality}p across all video tracks"
)
self.cdm = quality_based_cdm
dl_start_time = time.time()
if skip_dl:
@@ -1135,8 +1227,13 @@ class dl:
with Live(Padding(progress, (0, 5, 1, 5)), console=console):
for task_id, task_tracks in multiplex_tasks:
progress.start_task(task_id) # TODO: Needed?
audio_expected = not video_only and not no_audio
muxed_path, return_code, errors = task_tracks.mux(
str(title), progress=partial(progress.update, task_id=task_id), delete=False
str(title),
progress=partial(progress.update, task_id=task_id),
delete=False,
audio_expected=audio_expected,
title_language=title.language,
)
muxed_paths.append(muxed_path)
if return_code >= 2:
@@ -1209,6 +1306,9 @@ class dl:
if not drm:
return
if isinstance(track, Video) and track.height:
pass
if isinstance(drm, Widevine):
if not isinstance(self.cdm, (WidevineCdm, DecryptLabsRemoteCDM)) or (
isinstance(self.cdm, DecryptLabsRemoteCDM) and self.cdm.is_playready
@@ -1217,6 +1317,7 @@ class dl:
if widevine_cdm:
self.log.info("Switching to Widevine CDM for Widevine content")
self.cdm = widevine_cdm
elif isinstance(drm, PlayReady):
if not isinstance(self.cdm, (PlayReadyCdm, DecryptLabsRemoteCDM)) or (
isinstance(self.cdm, DecryptLabsRemoteCDM) and not self.cdm.is_playready
@@ -1228,14 +1329,20 @@ class dl:
if isinstance(drm, Widevine):
with self.DRM_TABLE_LOCK:
cek_tree = Tree(Text.assemble(("Widevine", "cyan"), (f"({drm.pssh.dumps()})", "text"), overflow="fold"))
pssh_display = self._truncate_pssh_for_display(drm.pssh.dumps(), "Widevine")
cek_tree = Tree(Text.assemble(("Widevine", "cyan"), (f"({pssh_display})", "text"), overflow="fold"))
pre_existing_tree = next(
(x for x in table.columns[0].cells if isinstance(x, Tree) and x.label == cek_tree.label), None
)
if pre_existing_tree:
cek_tree = pre_existing_tree
for kid in drm.kids:
need_license = False
all_kids = list(drm.kids)
if track_kid and track_kid not in all_kids:
all_kids.append(track_kid)
for kid in all_kids:
if kid in drm.content_keys:
continue
@@ -1255,46 +1362,51 @@ class dl:
if not pre_existing_tree:
table.add_row(cek_tree)
raise Widevine.Exceptions.CEKNotFound(msg)
else:
need_license = True
if kid not in drm.content_keys and not vaults_only:
from_vaults = drm.content_keys.copy()
if kid not in drm.content_keys and cdm_only:
need_license = True
try:
if self.service == "NF":
drm.get_NF_content_keys(cdm=self.cdm, licence=licence, certificate=certificate)
else:
drm.get_content_keys(cdm=self.cdm, licence=licence, certificate=certificate)
except Exception as e:
if isinstance(e, (Widevine.Exceptions.EmptyLicense, Widevine.Exceptions.CEKNotFound)):
msg = str(e)
else:
msg = f"An exception occurred in the Service's license function: {e}"
cek_tree.add(f"[logging.level.error]{msg}")
if not pre_existing_tree:
table.add_row(cek_tree)
raise e
if need_license and not vaults_only:
from_vaults = drm.content_keys.copy()
for kid_, key in drm.content_keys.items():
if key == "0" * 32:
key = f"[red]{key}[/]"
label = f"[text2]{kid_.hex}:{key}{is_track_kid}"
if not any(f"{kid_.hex}:{key}" in x.label for x in cek_tree.children):
cek_tree.add(label)
try:
if self.service == "NF":
drm.get_NF_content_keys(cdm=self.cdm, licence=licence, certificate=certificate)
else:
drm.get_content_keys(cdm=self.cdm, licence=licence, certificate=certificate)
except Exception as e:
if isinstance(e, (Widevine.Exceptions.EmptyLicense, Widevine.Exceptions.CEKNotFound)):
msg = str(e)
else:
msg = f"An exception occurred in the Service's license function: {e}"
cek_tree.add(f"[logging.level.error]{msg}")
if not pre_existing_tree:
table.add_row(cek_tree)
raise e
drm.content_keys = {
kid_: key for kid_, key in drm.content_keys.items() if key and key.count("0") != len(key)
}
for kid_, key in drm.content_keys.items():
if key == "0" * 32:
key = f"[red]{key}[/]"
is_track_kid_marker = ["", "*"][kid_ == track_kid]
label = f"[text2]{kid_.hex}:{key}{is_track_kid_marker}"
if not any(f"{kid_.hex}:{key}" in x.label for x in cek_tree.children):
cek_tree.add(label)
# The CDM keys may have returned blank content keys for KIDs we got from vaults.
# So we re-add the keys from vaults earlier overwriting blanks or removed KIDs data.
drm.content_keys.update(from_vaults)
drm.content_keys = {
kid_: key for kid_, key in drm.content_keys.items() if key and key.count("0") != len(key)
}
successful_caches = self.vaults.add_keys(drm.content_keys)
self.log.info(
f"Cached {len(drm.content_keys)} Key{'' if len(drm.content_keys) == 1 else 's'} to "
f"{successful_caches}/{len(self.vaults)} Vaults"
)
break # licensing twice will be unnecessary
# The CDM keys may have returned blank content keys for KIDs we got from vaults.
# So we re-add the keys from vaults earlier overwriting blanks or removed KIDs data.
drm.content_keys.update(from_vaults)
successful_caches = self.vaults.add_keys(drm.content_keys)
self.log.info(
f"Cached {len(drm.content_keys)} Key{'' if len(drm.content_keys) == 1 else 's'} to "
f"{successful_caches}/{len(self.vaults)} Vaults"
)
if track_kid and track_kid not in drm.content_keys:
msg = f"No Content Key for KID {track_kid.hex} was returned in the License"
@@ -1320,10 +1432,11 @@ class dl:
elif isinstance(drm, PlayReady):
with self.DRM_TABLE_LOCK:
pssh_display = self._truncate_pssh_for_display(drm.pssh_b64 or "", "PlayReady")
cek_tree = Tree(
Text.assemble(
("PlayReady", "cyan"),
(f"({drm.pssh_b64 or ''})", "text"),
(f"({pssh_display})", "text"),
overflow="fold",
)
)
@@ -1333,7 +1446,12 @@ class dl:
if pre_existing_tree:
cek_tree = pre_existing_tree
for kid in drm.kids:
need_license = False
all_kids = list(drm.kids)
if track_kid and track_kid not in all_kids:
all_kids.append(track_kid)
for kid in all_kids:
if kid in drm.content_keys:
continue
@@ -1353,35 +1471,40 @@ class dl:
if not pre_existing_tree:
table.add_row(cek_tree)
raise PlayReady.Exceptions.CEKNotFound(msg)
else:
need_license = True
if kid not in drm.content_keys and not vaults_only:
from_vaults = drm.content_keys.copy()
if kid not in drm.content_keys and cdm_only:
need_license = True
try:
drm.get_content_keys(cdm=self.cdm, licence=licence, certificate=certificate)
except Exception as e:
if isinstance(e, (PlayReady.Exceptions.EmptyLicense, PlayReady.Exceptions.CEKNotFound)):
msg = str(e)
else:
msg = f"An exception occurred in the Service's license function: {e}"
cek_tree.add(f"[logging.level.error]{msg}")
if not pre_existing_tree:
table.add_row(cek_tree)
raise e
if need_license and not vaults_only:
from_vaults = drm.content_keys.copy()
for kid_, key in drm.content_keys.items():
label = f"[text2]{kid_.hex}:{key}{is_track_kid}"
if not any(f"{kid_.hex}:{key}" in x.label for x in cek_tree.children):
cek_tree.add(label)
try:
drm.get_content_keys(cdm=self.cdm, licence=licence, certificate=certificate)
except Exception as e:
if isinstance(e, (PlayReady.Exceptions.EmptyLicense, PlayReady.Exceptions.CEKNotFound)):
msg = str(e)
else:
msg = f"An exception occurred in the Service's license function: {e}"
cek_tree.add(f"[logging.level.error]{msg}")
if not pre_existing_tree:
table.add_row(cek_tree)
raise e
drm.content_keys.update(from_vaults)
for kid_, key in drm.content_keys.items():
is_track_kid_marker = ["", "*"][kid_ == track_kid]
label = f"[text2]{kid_.hex}:{key}{is_track_kid_marker}"
if not any(f"{kid_.hex}:{key}" in x.label for x in cek_tree.children):
cek_tree.add(label)
successful_caches = self.vaults.add_keys(drm.content_keys)
self.log.info(
f"Cached {len(drm.content_keys)} Key{'' if len(drm.content_keys) == 1 else 's'} to "
f"{successful_caches}/{len(self.vaults)} Vaults"
)
break
drm.content_keys.update(from_vaults)
successful_caches = self.vaults.add_keys(drm.content_keys)
self.log.info(
f"Cached {len(drm.content_keys)} Key{'' if len(drm.content_keys) == 1 else 's'} to "
f"{successful_caches}/{len(self.vaults)} Vaults"
)
if track_kid and track_kid not in drm.content_keys:
msg = f"No Content Key for KID {track_kid.hex} was returned in the License"
@@ -1441,6 +1564,9 @@ class dl:
@staticmethod
def save_cookies(path: Path, cookies: CookieJar):
if hasattr(cookies, 'jar'):
cookies = cookies.jar
cookie_jar = MozillaCookieJar(path)
cookie_jar.load()
for cookie in cookies:
@@ -1467,9 +1593,11 @@ class dl:
service: str,
profile: Optional[str] = None,
drm: Optional[str] = None,
quality: Optional[int] = None,
) -> Optional[object]:
"""
Get CDM for a specified service (either Local or Remote CDM).
Now supports quality-based selection when quality is provided.
Raises a ValueError if there's a problem getting a CDM.
"""
cdm_name = config.cdm.get(service) or config.cdm.get("default")
@@ -1477,31 +1605,99 @@ class dl:
return None
if isinstance(cdm_name, dict):
lower_keys = {k.lower(): v for k, v in cdm_name.items()}
if {"widevine", "playready"} & lower_keys.keys():
drm_key = None
if drm:
drm_key = {
"wv": "widevine",
"widevine": "widevine",
"pr": "playready",
"playready": "playready",
}.get(drm.lower())
cdm_name = lower_keys.get(drm_key or "widevine") or lower_keys.get("playready")
else:
if not profile:
return None
cdm_name = cdm_name.get(profile) or config.cdm.get("default")
if not cdm_name:
return None
if quality:
quality_match = None
quality_keys = []
cdm_api = next(iter(x for x in config.remote_cdm if x["name"] == cdm_name), None)
for key in cdm_name.keys():
if (
isinstance(key, str)
and any(op in key for op in [">=", ">", "<=", "<"])
or (isinstance(key, str) and key.isdigit())
):
quality_keys.append(key)
def sort_quality_key(key):
if key.isdigit():
return (0, int(key)) # Exact matches first
elif key.startswith(">="):
return (1, -int(key[2:])) # >= descending
elif key.startswith(">"):
return (1, -int(key[1:])) # > descending
elif key.startswith("<="):
return (2, int(key[2:])) # <= ascending
elif key.startswith("<"):
return (2, int(key[1:])) # < ascending
return (3, 0) # Other keys last
quality_keys.sort(key=sort_quality_key)
for key in quality_keys:
if key.isdigit() and quality == int(key):
quality_match = cdm_name[key]
self.log.debug(f"Selected CDM based on exact quality match {quality}p: {quality_match}")
break
elif key.startswith(">="):
threshold = int(key[2:])
if quality >= threshold:
quality_match = cdm_name[key]
self.log.debug(f"Selected CDM based on quality {quality}p >= {threshold}p: {quality_match}")
break
elif key.startswith(">"):
threshold = int(key[1:])
if quality > threshold:
quality_match = cdm_name[key]
self.log.debug(f"Selected CDM based on quality {quality}p > {threshold}p: {quality_match}")
break
elif key.startswith("<="):
threshold = int(key[2:])
if quality <= threshold:
quality_match = cdm_name[key]
self.log.debug(f"Selected CDM based on quality {quality}p <= {threshold}p: {quality_match}")
break
elif key.startswith("<"):
threshold = int(key[1:])
if quality < threshold:
quality_match = cdm_name[key]
self.log.debug(f"Selected CDM based on quality {quality}p < {threshold}p: {quality_match}")
break
if quality_match:
cdm_name = quality_match
if isinstance(cdm_name, dict):
lower_keys = {k.lower(): v for k, v in cdm_name.items()}
if {"widevine", "playready"} & lower_keys.keys():
drm_key = None
if drm:
drm_key = {
"wv": "widevine",
"widevine": "widevine",
"pr": "playready",
"playready": "playready",
}.get(drm.lower())
cdm_name = lower_keys.get(drm_key or "widevine") or lower_keys.get("playready")
else:
cdm_name = cdm_name.get(profile) or cdm_name.get("default") or config.cdm.get("default")
if not cdm_name:
return None
cdm_api = next(iter(x.copy() for x in config.remote_cdm if x["name"] == cdm_name), None)
if cdm_api:
is_decrypt_lab = True if cdm_api.get("type") == "decrypt_labs" else False
if is_decrypt_lab:
del cdm_api["name"]
del cdm_api["type"]
if "secret" not in cdm_api or not cdm_api["secret"]:
if config.decrypt_labs_api_key:
cdm_api["secret"] = config.decrypt_labs_api_key
else:
raise ValueError(
f"No secret provided for DecryptLabs CDM '{cdm_name}' and no global "
"decrypt_labs_api_key configured"
)
# All DecryptLabs CDMs use DecryptLabsRemoteCDM
return DecryptLabsRemoteCDM(service_name=service, vaults=self.vaults, **cdm_api)
else:

View File

@@ -1 +1 @@
__version__ = "1.4.4"
__version__ = "1.4.7"

View File

@@ -6,9 +6,11 @@ from typing import Any, Dict, List, Optional, Union
from uuid import UUID
import requests
from pywidevine.cdm import Cdm as WidevineCdm
from pywidevine.device import DeviceTypes
from requests import Session
from unshackle.core import __version__
from unshackle.core.vaults import Vaults
@@ -79,15 +81,17 @@ class DecryptLabsRemoteCDM:
Key Features:
- Compatible with both Widevine and PlayReady DRM schemes
- Intelligent caching that compares required vs. available keys
- Optimized caching for L1/L2 devices (leverages API auto-optimization)
- Automatic key combination for mixed cache/license scenarios
- Seamless fallback to license requests when keys are missing
Intelligent Caching System:
1. DRM classes (PlayReady/Widevine) provide required KIDs via set_required_kids()
2. get_license_challenge() first checks for cached keys
3. If cached keys satisfy requirements, returns empty challenge (no license needed)
4. If keys are missing, makes targeted license request for remaining keys
5. parse_license() combines cached and license keys intelligently
3. For L1/L2 devices, always attempts cached keys first (API optimized)
4. If cached keys satisfy requirements, returns empty challenge (no license needed)
5. If keys are missing, makes targeted license request for remaining keys
6. parse_license() combines cached and license keys intelligently
"""
service_certificate_challenge = b"\x08\x04"
@@ -147,7 +151,7 @@ class DecryptLabsRemoteCDM:
{
"decrypt-labs-api-key": self.secret,
"Content-Type": "application/json",
"User-Agent": "unshackle-decrypt-labs-cdm/1.0",
"User-Agent": f"unshackle-decrypt-labs-cdm/{__version__}",
}
)
@@ -250,12 +254,14 @@ class DecryptLabsRemoteCDM:
"pssh": None,
"challenge": None,
"decrypt_labs_session_id": None,
"tried_cache": False,
"cached_keys": None,
}
return session_id
def close(self, session_id: bytes) -> None:
"""
Close a CDM session.
Close a CDM session and perform comprehensive cleanup.
Args:
session_id: Session identifier
@@ -266,6 +272,8 @@ class DecryptLabsRemoteCDM:
if session_id not in self._sessions:
raise DecryptLabsRemoteCDMExceptions.InvalidSession(f"Invalid session ID: {session_id.hex()}")
session = self._sessions[session_id]
session.clear()
del self._sessions[session_id]
def get_service_certificate(self, session_id: bytes) -> Optional[bytes]:
@@ -304,8 +312,13 @@ class DecryptLabsRemoteCDM:
raise DecryptLabsRemoteCDMExceptions.InvalidSession(f"Invalid session ID: {session_id.hex()}")
if certificate is None:
self._sessions[session_id]["service_certificate"] = None
return "Removed"
if not self._is_playready and self.device_name == "L1":
certificate = WidevineCdm.common_privacy_cert
self._sessions[session_id]["service_certificate"] = base64.b64decode(certificate)
return "Using default Widevine common privacy certificate for L1"
else:
self._sessions[session_id]["service_certificate"] = None
return "No certificate set (not required for this device type)"
if isinstance(certificate, str):
certificate = base64.b64decode(certificate)
@@ -340,15 +353,19 @@ class DecryptLabsRemoteCDM:
Generate a license challenge using Decrypt Labs API with intelligent caching.
This method implements smart caching logic that:
1. First attempts to retrieve cached keys from the API
2. If required KIDs are set, compares cached keys against requirements
3. Only makes a license request if keys are missing
4. Returns empty challenge if all required keys are cached
1. First checks local vaults for required keys
2. Attempts to retrieve cached keys from the API
3. If required KIDs are set, compares available keys (vault + cached) against requirements
4. Only makes a license request if keys are missing
5. Returns empty challenge if all required keys are available
The intelligent caching works as follows:
- Local vaults: Always checked first if available
- For L1/L2 devices: Always prioritizes cached keys (API automatically optimizes)
- For other devices: Uses cache retry logic based on session state
- With required KIDs set: Only requests license for missing keys
- Without required KIDs: Returns any available cached keys
- For PlayReady: Combines cached keys with license keys seamlessly
- For PlayReady: Combines vault, cached, and license keys seamlessly
Args:
session_id: Session identifier
@@ -357,7 +374,7 @@ class DecryptLabsRemoteCDM:
privacy_mode: Whether to use privacy mode - for compatibility only
Returns:
License challenge as bytes, or empty bytes if cached keys satisfy requirements
License challenge as bytes, or empty bytes if available keys satisfy requirements
Raises:
InvalidSession: If session ID is invalid
@@ -365,6 +382,8 @@ class DecryptLabsRemoteCDM:
Note:
Call set_required_kids() before this method for optimal caching behavior.
L1/L2 devices automatically use cached keys when available per API design.
Local vault keys are always checked first when vaults are available.
"""
_ = license_type, privacy_mode
@@ -377,13 +396,43 @@ class DecryptLabsRemoteCDM:
init_data = self._get_init_data_from_pssh(pssh_or_wrm)
already_tried_cache = session.get("tried_cache", False)
if self.vaults and self._required_kids:
vault_keys = []
for kid_str in self._required_kids:
try:
clean_kid = kid_str.replace("-", "")
if len(clean_kid) == 32:
kid_uuid = UUID(hex=clean_kid)
else:
kid_uuid = UUID(hex=clean_kid.ljust(32, "0"))
key, _ = self.vaults.get_key(kid_uuid)
if key and key.count("0") != len(key):
vault_keys.append({"kid": kid_str, "key": key, "type": "CONTENT"})
except (ValueError, TypeError):
continue
if vault_keys:
vault_kids = set(k["kid"] for k in vault_keys)
required_kids = set(self._required_kids)
if required_kids.issubset(vault_kids):
session["keys"] = vault_keys
return b""
else:
session["vault_keys"] = vault_keys
if self.device_name in ["L1", "L2"]:
get_cached_keys = True
else:
get_cached_keys = not already_tried_cache
request_data = {
"scheme": self.device_name,
"init_data": init_data,
"get_cached_keys_if_exists": not already_tried_cache,
"get_cached_keys_if_exists": get_cached_keys,
}
if self.device_name in ["L1", "L2", "SL2", "SL3"] and self.service_name:
if self.service_name:
request_data["service"] = self.service_name
if session["service_certificate"]:
@@ -420,22 +469,49 @@ class DecryptLabsRemoteCDM:
"""
cached_keys = data.get("cached_keys", [])
parsed_keys = self._parse_cached_keys(cached_keys)
session["keys"] = parsed_keys
all_available_keys = list(parsed_keys)
if "vault_keys" in session:
all_available_keys.extend(session["vault_keys"])
session["keys"] = all_available_keys
session["tried_cache"] = True
if self._required_kids:
cached_kids = set()
for key in parsed_keys:
available_kids = set()
for key in all_available_keys:
if isinstance(key, dict) and "kid" in key:
cached_kids.add(key["kid"].replace("-", "").lower())
available_kids.add(key["kid"].replace("-", "").lower())
required_kids = set(self._required_kids)
missing_kids = required_kids - cached_kids
missing_kids = required_kids - available_kids
if missing_kids:
session["cached_keys"] = parsed_keys
request_data["get_cached_keys_if_exists"] = False
response = self._http_session.post(f"{self.host}/get-request", json=request_data, timeout=30)
if self.device_name in ["L1", "L2"]:
license_request_data = {
"scheme": self.device_name,
"init_data": init_data,
"get_cached_keys_if_exists": False,
}
if self.service_name:
license_request_data["service"] = self.service_name
if session["service_certificate"]:
license_request_data["service_certificate"] = base64.b64encode(
session["service_certificate"]
).decode("utf-8")
else:
license_request_data = request_data.copy()
license_request_data["get_cached_keys_if_exists"] = False
session["decrypt_labs_session_id"] = None
session["challenge"] = None
session["tried_cache"] = False
response = self._http_session.post(
f"{self.host}/get-request", json=license_request_data, timeout=30
)
if response.status_code == 200:
data = response.json()
if data.get("message") == "success" and "challenge" in data:
@@ -542,50 +618,63 @@ class DecryptLabsRemoteCDM:
license_keys = self._parse_keys_response(data)
if self.is_playready and "cached_keys" in session:
"""
Combine cached keys with license keys for PlayReady content.
all_keys = []
This ensures we have both the cached keys (obtained earlier) and
any additional keys from the license response, without duplicates.
"""
if "vault_keys" in session:
all_keys.extend(session["vault_keys"])
if "cached_keys" in session:
cached_keys = session.get("cached_keys", [])
all_keys = list(cached_keys)
if cached_keys:
for cached_key in cached_keys:
all_keys.append(cached_key)
for license_key in license_keys:
already_exists = False
license_kid = None
if isinstance(license_key, dict) and "kid" in license_key:
license_kid = license_key["kid"].replace("-", "").lower()
elif hasattr(license_key, "kid"):
license_kid = str(license_key.kid).replace("-", "").lower()
elif hasattr(license_key, "key_id"):
license_kid = str(license_key.key_id).replace("-", "").lower()
for license_key in license_keys:
already_exists = False
license_kid = None
if isinstance(license_key, dict) and "kid" in license_key:
license_kid = license_key["kid"].replace("-", "").lower()
elif hasattr(license_key, "kid"):
license_kid = str(license_key.kid).replace("-", "").lower()
elif hasattr(license_key, "key_id"):
license_kid = str(license_key.key_id).replace("-", "").lower()
if license_kid:
for cached_key in cached_keys:
cached_kid = None
if isinstance(cached_key, dict) and "kid" in cached_key:
cached_kid = cached_key["kid"].replace("-", "").lower()
elif hasattr(cached_key, "kid"):
cached_kid = str(cached_key.kid).replace("-", "").lower()
elif hasattr(cached_key, "key_id"):
cached_kid = str(cached_key.key_id).replace("-", "").lower()
if license_kid:
for existing_key in all_keys:
existing_kid = None
if isinstance(existing_key, dict) and "kid" in existing_key:
existing_kid = existing_key["kid"].replace("-", "").lower()
elif hasattr(existing_key, "kid"):
existing_kid = str(existing_key.kid).replace("-", "").lower()
elif hasattr(existing_key, "key_id"):
existing_kid = str(existing_key.key_id).replace("-", "").lower()
if cached_kid == license_kid:
already_exists = True
break
if existing_kid == license_kid:
already_exists = True
break
if not already_exists:
all_keys.append(license_key)
if not already_exists:
all_keys.append(license_key)
session["keys"] = all_keys
else:
session["keys"] = license_keys
session["keys"] = all_keys
session.pop("cached_keys", None)
session.pop("vault_keys", None)
if self.vaults and session["keys"]:
key_dict = {UUID(hex=key["kid"]): key["key"] for key in session["keys"] if key["type"] == "CONTENT"}
self.vaults.add_keys(key_dict)
key_dict = {}
for key in session["keys"]:
if key["type"] == "CONTENT":
try:
clean_kid = key["kid"].replace("-", "")
if len(clean_kid) == 32:
kid_uuid = UUID(hex=clean_kid)
else:
kid_uuid = UUID(hex=clean_kid.ljust(32, "0"))
key_dict[kid_uuid] = key["key"]
except (ValueError, TypeError):
continue
if key_dict:
self.vaults.add_keys(key_dict)
def get_keys(self, session_id: bytes, type_: Optional[str] = None) -> List[Key]:
"""

View File

@@ -88,6 +88,7 @@ class Config:
self.tag_group_name: bool = kwargs.get("tag_group_name", True)
self.tag_imdb_tmdb: bool = kwargs.get("tag_imdb_tmdb", True)
self.tmdb_api_key: str = kwargs.get("tmdb_api_key") or ""
self.decrypt_labs_api_key: str = kwargs.get("decrypt_labs_api_key") or ""
self.update_checks: bool = kwargs.get("update_checks", True)
self.update_check_interval: int = kwargs.get("update_check_interval", 24)
self.scene_naming: bool = kwargs.get("scene_naming", True)

View File

@@ -150,6 +150,7 @@ def download(
track_type = track.__class__.__name__
thread_count = str(config.n_m3u8dl_re.get("thread_count", max_workers))
retry_count = str(config.n_m3u8dl_re.get("retry_count", max_workers))
ad_keyword = config.n_m3u8dl_re.get("ad_keyword")
arguments = [
@@ -160,6 +161,8 @@ def download(
output_dir,
"--thread-count",
thread_count,
"--download-retry-count",
retry_count,
"--no-log",
"--write-meta-json",
"false",

View File

@@ -8,6 +8,7 @@ from urllib.parse import urljoin
from Cryptodome.Cipher import AES
from Cryptodome.Util.Padding import unpad
from curl_cffi.requests import Session as CurlSession
from m3u8.model import Key
from requests import Session
@@ -69,8 +70,8 @@ class ClearKey:
"""
if not isinstance(m3u_key, Key):
raise ValueError(f"Provided M3U Key is in an unexpected type {m3u_key!r}")
if not isinstance(session, (Session, type(None))):
raise TypeError(f"Expected session to be a {Session}, not a {type(session)}")
if not isinstance(session, (Session, CurlSession, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not a {type(session)}")
if not m3u_key.method.startswith("AES"):
raise ValueError(f"Provided M3U Key is not an AES Clear Key, {m3u_key.method}")

View File

@@ -15,6 +15,7 @@ from uuid import UUID
from zlib import crc32
import requests
from curl_cffi.requests import Session as CurlSession
from langcodes import Language, tag_is_valid
from lxml.etree import Element, ElementTree
from pyplayready.system.pssh import PSSH as PR_PSSH
@@ -47,7 +48,7 @@ class DASH:
self.url = url
@classmethod
def from_url(cls, url: str, session: Optional[Session] = None, **args: Any) -> DASH:
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession]] = None, **args: Any) -> DASH:
if not url:
raise requests.URLRequired("DASH manifest URL must be provided for relative path computations.")
if not isinstance(url, str):
@@ -55,8 +56,8 @@ class DASH:
if not session:
session = Session()
elif not isinstance(session, Session):
raise TypeError(f"Expected session to be a {Session}, not {session!r}")
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
res = session.get(url, **args)
if res.url != url:
@@ -103,6 +104,10 @@ class DASH:
continue
if next(iter(period.xpath("SegmentType/@value")), "content") != "content":
continue
if "urn:amazon:primevideo:cachingBreadth" in [
x.get("schemeIdUri") for x in period.findall("SupplementalProperty")
]:
continue
for adaptation_set in period.findall("AdaptationSet"):
if self.is_trick_mode(adaptation_set):

View File

@@ -14,9 +14,10 @@ from typing import Any, Callable, Optional, Union
from urllib.parse import urljoin
from zlib import crc32
import httpx
import m3u8
import requests
from curl_cffi.requests import Response as CurlResponse
from curl_cffi.requests import Session as CurlSession
from langcodes import Language, tag_is_valid
from m3u8 import M3U8
from pyplayready.cdm import Cdm as PlayReadyCdm
@@ -35,7 +36,7 @@ from unshackle.core.utilities import get_extension, is_close_match, try_ensure_u
class HLS:
def __init__(self, manifest: M3U8, session: Optional[Union[Session, httpx.Client]] = None):
def __init__(self, manifest: M3U8, session: Optional[Union[Session, CurlSession]] = None):
if not manifest:
raise ValueError("HLS manifest must be provided.")
if not isinstance(manifest, M3U8):
@@ -47,7 +48,7 @@ class HLS:
self.session = session or Session()
@classmethod
def from_url(cls, url: str, session: Optional[Union[Session, httpx.Client]] = None, **args: Any) -> HLS:
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession]] = None, **args: Any) -> HLS:
if not url:
raise requests.URLRequired("HLS manifest URL must be provided.")
if not isinstance(url, str):
@@ -55,22 +56,22 @@ class HLS:
if not session:
session = Session()
elif not isinstance(session, (Session, httpx.Client)):
raise TypeError(f"Expected session to be a {Session} or {httpx.Client}, not {session!r}")
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
res = session.get(url, **args)
# Handle both requests and httpx response objects
# Handle requests and curl_cffi response objects
if isinstance(res, requests.Response):
if not res.ok:
raise requests.ConnectionError("Failed to request the M3U(8) document.", response=res)
content = res.text
elif isinstance(res, httpx.Response):
if res.status_code >= 400:
elif isinstance(res, CurlResponse):
if not res.ok:
raise requests.ConnectionError("Failed to request the M3U(8) document.", response=res)
content = res.text
else:
raise TypeError(f"Expected response to be a requests.Response or httpx.Response, not {type(res)}")
raise TypeError(f"Expected response to be a requests.Response or curl_cffi.Response, not {type(res)}")
master = m3u8.loads(content, uri=url)
@@ -229,7 +230,7 @@ class HLS:
save_path: Path,
save_dir: Path,
progress: partial,
session: Optional[Union[Session, httpx.Client]] = None,
session: Optional[Union[Session, CurlSession]] = None,
proxy: Optional[str] = None,
max_workers: Optional[int] = None,
license_widevine: Optional[Callable] = None,
@@ -238,15 +239,13 @@ class HLS:
) -> None:
if not session:
session = Session()
elif not isinstance(session, (Session, httpx.Client)):
raise TypeError(f"Expected session to be a {Session} or {httpx.Client}, not {session!r}")
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
if proxy:
# Handle proxies differently based on session type
if isinstance(session, Session):
session.proxies.update({"all": proxy})
elif isinstance(session, httpx.Client):
session.proxies = {"http://": proxy, "https://": proxy}
log = logging.getLogger("HLS")
@@ -257,13 +256,8 @@ class HLS:
log.error(f"Failed to request the invariant M3U8 playlist: {response.status_code}")
sys.exit(1)
playlist_text = response.text
elif isinstance(response, httpx.Response):
if response.status_code >= 400:
log.error(f"Failed to request the invariant M3U8 playlist: {response.status_code}")
sys.exit(1)
playlist_text = response.text
else:
raise TypeError(f"Expected response to be a requests.Response or httpx.Response, not {type(response)}")
raise TypeError(f"Expected response to be a requests.Response or curl_cffi.Response, not {type(response)}")
master = m3u8.loads(playlist_text, uri=track.url)
@@ -533,13 +527,9 @@ class HLS:
if isinstance(res, requests.Response):
res.raise_for_status()
init_content = res.content
elif isinstance(res, httpx.Response):
if res.status_code >= 400:
raise requests.HTTPError(f"HTTP Error: {res.status_code}", response=res)
init_content = res.content
else:
raise TypeError(
f"Expected response to be requests.Response or httpx.Response, not {type(res)}"
f"Expected response to be requests.Response or curl_cffi.Response, not {type(res)}"
)
map_data = (segment.init_section, init_content)
@@ -707,7 +697,7 @@ class HLS:
@staticmethod
def parse_session_data_keys(
manifest: M3U8, session: Optional[Union[Session, httpx.Client]] = None
manifest: M3U8, session: Optional[Union[Session, CurlSession]] = None
) -> list[m3u8.model.Key]:
"""Parse `com.apple.hls.keys` session data and return Key objects."""
keys: list[m3u8.model.Key] = []
@@ -798,7 +788,8 @@ class HLS:
@staticmethod
def get_drm(
key: Union[m3u8.model.SessionKey, m3u8.model.Key], session: Optional[Union[Session, httpx.Client]] = None
key: Union[m3u8.model.SessionKey, m3u8.model.Key],
session: Optional[Union[Session, CurlSession]] = None,
) -> DRM_T:
"""
Convert HLS EXT-X-KEY data to an initialized DRM object.
@@ -810,8 +801,8 @@ class HLS:
Raises a NotImplementedError if the key system is not supported.
"""
if not isinstance(session, (Session, httpx.Client, type(None))):
raise TypeError(f"Expected session to be a {Session} or {httpx.Client}, not {type(session)}")
if not isinstance(session, (Session, CurlSession, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {type(session)}")
if not session:
session = Session()

View File

@@ -10,6 +10,7 @@ from pathlib import Path
from typing import Any, Callable, Optional, Union
import requests
from curl_cffi.requests import Session as CurlSession
from langcodes import Language, tag_is_valid
from lxml.etree import Element
from pyplayready.system.pssh import PSSH as PR_PSSH
@@ -34,11 +35,13 @@ class ISM:
self.url = url
@classmethod
def from_url(cls, url: str, session: Optional[Session] = None, **kwargs: Any) -> "ISM":
def from_url(cls, url: str, session: Optional[Union[Session, CurlSession]] = None, **kwargs: Any) -> "ISM":
if not url:
raise requests.URLRequired("ISM manifest URL must be provided")
if not session:
session = Session()
elif not isinstance(session, (Session, CurlSession)):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {session!r}")
res = session.get(url, **kwargs)
if res.url != url:
url = res.url

View File

@@ -4,15 +4,10 @@ from __future__ import annotations
from typing import Optional, Union
import httpx
import m3u8
from pyplayready.cdm import Cdm as PlayReadyCdm
from pyplayready.system.pssh import PSSH as PR_PSSH
from pywidevine.cdm import Cdm as WidevineCdm
from pywidevine.pssh import PSSH as WV_PSSH
from curl_cffi.requests import Session as CurlSession
from requests import Session
from unshackle.core.drm import PlayReady, Widevine
from unshackle.core.manifests.hls import HLS
from unshackle.core.tracks import Tracks
@@ -21,54 +16,17 @@ def parse(
master: m3u8.M3U8,
language: str,
*,
session: Optional[Union[Session, httpx.Client]] = None,
session: Optional[Union[Session, CurlSession]] = None,
) -> Tracks:
"""Parse a variant playlist to ``Tracks`` with DRM information."""
"""Parse a variant playlist to ``Tracks`` with basic information, defer DRM loading."""
tracks = HLS(master, session=session).to_tracks(language)
need_wv = not any(isinstance(d, Widevine) for t in tracks for d in (t.drm or []))
need_pr = not any(isinstance(d, PlayReady) for t in tracks for d in (t.drm or []))
bool(master.session_keys or HLS.parse_session_data_keys(master, session or Session()))
if (need_wv or need_pr) and tracks.videos:
if not session:
session = Session()
session_keys = list(master.session_keys or [])
session_keys.extend(HLS.parse_session_data_keys(master, session))
for drm_obj in HLS.get_all_drm(session_keys):
if need_wv and isinstance(drm_obj, Widevine):
for t in tracks.videos + tracks.audio:
t.drm = [d for d in (t.drm or []) if not isinstance(d, Widevine)] + [drm_obj]
need_wv = False
elif need_pr and isinstance(drm_obj, PlayReady):
for t in tracks.videos + tracks.audio:
t.drm = [d for d in (t.drm or []) if not isinstance(d, PlayReady)] + [drm_obj]
need_pr = False
if not need_wv and not need_pr:
break
if (need_wv or need_pr) and tracks.videos:
first_video = tracks.videos[0]
playlist = m3u8.load(first_video.url)
for key in playlist.keys or []:
if not key or not key.keyformat:
continue
fmt = key.keyformat.lower()
if need_wv and fmt == WidevineCdm.urn:
pssh_b64 = key.uri.split(",")[-1]
drm = Widevine(pssh=WV_PSSH(pssh_b64))
for t in tracks.videos + tracks.audio:
t.drm = [d for d in (t.drm or []) if not isinstance(d, Widevine)] + [drm]
need_wv = False
elif need_pr and (fmt == PlayReadyCdm or "com.microsoft.playready" in fmt):
pssh_b64 = key.uri.split(",")[-1]
drm = PlayReady(pssh=PR_PSSH(pssh_b64), pssh_b64=pssh_b64)
for t in tracks.videos + tracks.audio:
t.drm = [d for d in (t.drm or []) if not isinstance(d, PlayReady)] + [drm]
need_pr = False
if not need_wv and not need_pr:
break
if True:
for t in tracks.videos + tracks.audio:
t.needs_drm_loading = True
t.session = session
return tracks

79
unshackle/core/session.py Normal file
View File

@@ -0,0 +1,79 @@
"""Session utilities for creating HTTP sessions with different backends."""
from __future__ import annotations
import warnings
from curl_cffi.requests import Session as CurlSession
from unshackle.core.config import config
# Globally suppress curl_cffi HTTPS proxy warnings since some proxy providers
# (like NordVPN) require HTTPS URLs but curl_cffi expects HTTP format
warnings.filterwarnings(
"ignore", message="Make sure you are using https over https proxy.*", category=RuntimeWarning, module="curl_cffi.*"
)
class Session(CurlSession):
"""curl_cffi Session with warning suppression."""
def request(self, method, url, **kwargs):
with warnings.catch_warnings():
warnings.filterwarnings(
"ignore", message="Make sure you are using https over https proxy.*", category=RuntimeWarning
)
return super().request(method, url, **kwargs)
def session(browser: str | None = None, **kwargs) -> Session:
"""
Create a curl_cffi session that impersonates a browser.
This is a full replacement for requests.Session with browser impersonation
and anti-bot capabilities. The session uses curl-impersonate under the hood
to mimic real browser behavior.
Args:
browser: Browser to impersonate (e.g. "chrome124", "firefox", "safari").
Uses the configured default from curl_impersonate.browser if not specified.
See https://github.com/lexiforest/curl_cffi#sessions for available options.
**kwargs: Additional arguments passed to CurlSession constructor:
- headers: Additional headers (dict)
- cookies: Cookie jar or dict
- auth: HTTP basic auth tuple (username, password)
- proxies: Proxy configuration dict
- verify: SSL certificate verification (bool, default True)
- timeout: Request timeout in seconds (float or tuple)
- allow_redirects: Follow redirects (bool, default True)
- max_redirects: Maximum redirect count (int)
- cert: Client certificate (str or tuple)
Returns:
curl_cffi.requests.Session configured with browser impersonation, common headers,
and equivalent retry behavior to requests.Session.
Example:
from unshackle.core.session import session
class MyService(Service):
@staticmethod
def get_session():
return session() # Uses config default browser
"""
if browser is None:
browser = config.curl_impersonate.get("browser", "chrome124")
session_config = {
"impersonate": browser,
"timeout": 30.0,
"allow_redirects": True,
"max_redirects": 15,
"verify": True,
}
session_config.update(kwargs)
session_obj = Session(**session_config)
session_obj.headers.update(config.headers)
return session_obj

View File

@@ -13,6 +13,7 @@ from typing import Any, Callable, Iterable, Optional, Union
from uuid import UUID
from zlib import crc32
from curl_cffi.requests import Session as CurlSession
from langcodes import Language
from pyplayready.cdm import Cdm as PlayReadyCdm
from pywidevine.cdm import Cdm as WidevineCdm
@@ -420,7 +421,7 @@ class Track:
for drm in self.drm:
if isinstance(drm, PlayReady):
return drm
elif hasattr(cdm, 'is_playready'):
elif hasattr(cdm, "is_playready"):
if cdm.is_playready:
for drm in self.drm:
if isinstance(drm, PlayReady):
@@ -473,6 +474,83 @@ class Track:
if tenc.key_ID.int != 0:
return tenc.key_ID
def load_drm_if_needed(self, service=None) -> bool:
"""
Load DRM information for this track if it was deferred during parsing.
Args:
service: Service instance that can fetch track-specific DRM info
Returns:
True if DRM was loaded or already present, False if failed
"""
if not getattr(self, "needs_drm_loading", False):
return bool(self.drm)
if self.drm:
self.needs_drm_loading = False
return True
if not service or not hasattr(service, "get_track_drm"):
return self.load_drm_from_playlist()
try:
track_drm = service.get_track_drm(self)
if track_drm:
self.drm = track_drm if isinstance(track_drm, list) else [track_drm]
self.needs_drm_loading = False
return True
except Exception as e:
raise ValueError(f"Failed to load DRM from service for track {self.id}: {e}")
return self.load_drm_from_playlist()
def load_drm_from_playlist(self) -> bool:
"""
Fallback method to load DRM by fetching this track's individual playlist.
"""
if self.drm:
self.needs_drm_loading = False
return True
try:
import m3u8
from pyplayready.cdm import Cdm as PlayReadyCdm
from pyplayready.system.pssh import PSSH as PR_PSSH
from pywidevine.cdm import Cdm as WidevineCdm
from pywidevine.pssh import PSSH as WV_PSSH
session = getattr(self, "session", None) or Session()
response = session.get(self.url)
playlist = m3u8.loads(response.text, self.url)
drm_list = []
for key in playlist.keys or []:
if not key or not key.keyformat:
continue
fmt = key.keyformat.lower()
if fmt == WidevineCdm.urn:
pssh_b64 = key.uri.split(",")[-1]
drm = Widevine(pssh=WV_PSSH(pssh_b64))
drm_list.append(drm)
elif fmt == PlayReadyCdm or "com.microsoft.playready" in fmt:
pssh_b64 = key.uri.split(",")[-1]
drm = PlayReady(pssh=PR_PSSH(pssh_b64), pssh_b64=pssh_b64)
drm_list.append(drm)
if drm_list:
self.drm = drm_list
self.needs_drm_loading = False
return True
except Exception as e:
raise ValueError(f"Failed to load DRM from playlist for track {self.id}: {e}")
return False
def get_init_segment(
self,
maximum_size: int = 20000,
@@ -508,8 +586,8 @@ class Track:
raise TypeError(f"Expected url to be a {str}, not {type(url)}")
if not isinstance(byte_range, (str, type(None))):
raise TypeError(f"Expected byte_range to be a {str}, not {type(byte_range)}")
if not isinstance(session, (Session, type(None))):
raise TypeError(f"Expected session to be a {Session}, not {type(session)}")
if not isinstance(session, (Session, CurlSession, type(None))):
raise TypeError(f"Expected session to be a {Session} or {CurlSession}, not {type(session)}")
if not url:
if self.descriptor != self.Descriptor.URL:
@@ -567,15 +645,32 @@ class Track:
output_path = original_path.with_stem(f"{original_path.stem}_repack")
def _ffmpeg(extra_args: list[str] = None):
subprocess.run(
args = [
binaries.FFMPEG,
"-hide_banner",
"-loglevel",
"error",
"-i",
original_path,
*(extra_args or []),
]
if hasattr(self, "data") and self.data.get("audio_language"):
audio_lang = self.data["audio_language"]
audio_name = self.data.get("audio_language_name", audio_lang)
args.extend(
[
"-metadata:s:a:0",
f"language={audio_lang}",
"-metadata:s:a:0",
f"title={audio_name}",
"-metadata:s:a:0",
f"handler_name={audio_name}",
]
)
args.extend(
[
binaries.FFMPEG,
"-hide_banner",
"-loglevel",
"error",
"-i",
original_path,
*(extra_args or []),
# Following are very important!
"-map_metadata",
"-1", # don't transfer metadata to output file
@@ -584,7 +679,11 @@ class Track:
"-codec",
"copy",
str(output_path),
],
]
)
subprocess.run(
args,
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,

View File

@@ -181,7 +181,7 @@ class Tracks:
log = logging.getLogger("Tracks")
if duplicates:
log.warning(f" - Found and skipped {duplicates} duplicate tracks...")
log.debug(f" - Found and skipped {duplicates} duplicate tracks...")
def sort_videos(self, by_language: Optional[Sequence[Union[str, Language]]] = None) -> None:
"""Sort video tracks by bitrate, and optionally language."""
@@ -305,7 +305,14 @@ class Tracks:
)
return selected
def mux(self, title: str, delete: bool = True, progress: Optional[partial] = None) -> tuple[Path, int, list[str]]:
def mux(
self,
title: str,
delete: bool = True,
progress: Optional[partial] = None,
audio_expected: bool = True,
title_language: Optional[Language] = None,
) -> tuple[Path, int, list[str]]:
"""
Multiplex all the Tracks into a Matroska Container file.
@@ -315,7 +322,28 @@ class Tracks:
delete: Delete all track files after multiplexing.
progress: Update a rich progress bar via `completed=...`. This must be the
progress object's update() func, pre-set with task id via functools.partial.
audio_expected: Whether audio is expected in the output. Used to determine
if embedded audio metadata should be added.
title_language: The title's intended language. Used to select the best video track
for audio metadata when multiple video tracks exist.
"""
if self.videos and not self.audio and audio_expected:
video_track = None
if title_language:
video_track = next((v for v in self.videos if v.language == title_language), None)
if not video_track:
video_track = next((v for v in self.videos if v.is_original_lang), None)
video_track = video_track or self.videos[0]
if video_track.language.is_valid():
lang_code = str(video_track.language)
lang_name = video_track.language.display_name()
for video in self.videos:
video.needs_repack = True
video.data["audio_language"] = lang_code
video.data["audio_language_name"] = lang_name
if not binaries.MKVToolNix:
raise RuntimeError("MKVToolNix (mkvmerge) is required for muxing but was not found")
@@ -332,12 +360,20 @@ class Tracks:
raise ValueError("Video Track must be downloaded before muxing...")
events.emit(events.Types.TRACK_MULTIPLEX, track=vt)
is_default = False
if title_language:
is_default = vt.language == title_language
if not any(v.language == title_language for v in self.videos):
is_default = vt.is_original_lang or i == 0
else:
is_default = i == 0
# Prepare base arguments
video_args = [
"--language",
f"0:{vt.language}",
"--default-track",
f"0:{i == 0}",
f"0:{is_default}",
"--original-flag",
f"0:{vt.is_original_lang}",
"--compression",
@@ -363,6 +399,18 @@ class Tracks:
]
)
if hasattr(vt, "data") and vt.data.get("audio_language"):
audio_lang = vt.data["audio_language"]
audio_name = vt.data.get("audio_language_name", audio_lang)
video_args.extend(
[
"--language",
f"1:{audio_lang}",
"--track-name",
f"1:{audio_name}",
]
)
cl.extend(video_args + ["(", str(vt.path), ")"])
for i, at in enumerate(self.audio):

View File

@@ -74,7 +74,9 @@ class Vaults:
for vault in self.vaults:
if not vault.no_push:
try:
success += bool(vault.add_keys(self.service, kid_keys))
# Count each vault that successfully processes the keys (whether new or existing)
vault.add_keys(self.service, kid_keys)
success += 1
except (PermissionError, NotImplementedError):
pass
return success

View File

@@ -282,6 +282,10 @@ class EXAMPLE(Service):
return chapters
def get_widevine_service_certificate(self, **_: any) -> str:
"""Return the Widevine service certificate from config, if available."""
return self.config.get("certificate")
def get_playready_license(self, *, challenge: bytes, title: Title_T, track: AnyTrack) -> Optional[bytes]:
"""Retrieve a PlayReady license for a given track."""

View File

@@ -88,6 +88,26 @@ cdm:
jane_uhd: nexus_5_l1 # Profile 'jane_uhd' uses Nexus 5 L1
default: generic_android_l3 # Default CDM for this service
# NEW: Quality-based CDM selection
# Use different CDMs based on video resolution
# Supports operators: >=, >, <=, <, or exact match
EXAMPLE_QUALITY:
"<=1080": generic_android_l3 # Use L3 for 1080p and below
">1080": nexus_5_l1 # Use L1 for above 1080p (1440p, 2160p)
default: generic_android_l3 # Optional: fallback if no quality match
# You can mix profiles and quality thresholds in the same service
NETFLIX:
# Profile-based selection (existing functionality)
john: netflix_l3_profile
jane: netflix_l1_profile
# Quality-based selection (new functionality)
"<=720": netflix_mobile_l3
"1080": netflix_standard_l3
">=1440": netflix_premium_l1
# Fallback
default: netflix_standard_l3
# Use pywidevine Serve-compliant Remote CDMs
remote_cdm:
- name: "chrome"
@@ -106,16 +126,16 @@ remote_cdm:
secret: secret_key
- name: "decrypt_labs_chrome"
type: "decrypt_labs" # Required to identify as DecryptLabs CDM
device_name: "ChromeCDM" # Scheme identifier - must match exactly
type: "decrypt_labs" # Required to identify as DecryptLabs CDM
device_name: "ChromeCDM" # Scheme identifier - must match exactly
device_type: CHROME
system_id: 4464 # Doesn't matter
security_level: 3
host: "https://keyxtractor.decryptlabs.com"
secret: "your_decrypt_labs_api_key_here" # Replace with your API key
secret: "your_decrypt_labs_api_key_here" # Replace with your API key
- name: "decrypt_labs_l1"
type: "decrypt_labs"
device_name: "L1" # Scheme identifier - must match exactly
device_name: "L1" # Scheme identifier - must match exactly
device_type: ANDROID
system_id: 4464
security_level: 1
@@ -124,7 +144,7 @@ remote_cdm:
- name: "decrypt_labs_l2"
type: "decrypt_labs"
device_name: "L2" # Scheme identifier - must match exactly
device_name: "L2" # Scheme identifier - must match exactly
device_type: ANDROID
system_id: 4464
security_level: 2
@@ -133,7 +153,7 @@ remote_cdm:
- name: "decrypt_labs_playready_sl2"
type: "decrypt_labs"
device_name: "SL2" # Scheme identifier - must match exactly
device_name: "SL2" # Scheme identifier - must match exactly
device_type: PLAYREADY
system_id: 0
security_level: 2000
@@ -142,7 +162,7 @@ remote_cdm:
- name: "decrypt_labs_playready_sl3"
type: "decrypt_labs"
device_name: "SL3" # Scheme identifier - must match exactly
device_name: "SL3" # Scheme identifier - must match exactly
device_type: PLAYREADY
system_id: 0
security_level: 3000

View File

@@ -28,26 +28,33 @@ class MySQL(Vault):
raise PermissionError(f"MySQL vault {self.slug} has no SELECT permission.")
def get_key(self, kid: Union[UUID, str], service: str) -> Optional[str]:
if not self.has_table(service):
# no table, no key, simple
return None
if isinstance(kid, UUID):
kid = kid.hex
service_variants = [service]
if service != service.lower():
service_variants.append(service.lower())
if service != service.upper():
service_variants.append(service.upper())
conn = self.conn_factory.get()
cursor = conn.cursor()
try:
cursor.execute(
# TODO: SQL injection risk
f"SELECT `id`, `key_` FROM `{service}` WHERE `kid`=%s AND `key_`!=%s",
(kid, "0" * 32),
)
cek = cursor.fetchone()
if not cek:
return None
return cek["key_"]
for service_name in service_variants:
if not self.has_table(service_name):
continue
cursor.execute(
# TODO: SQL injection risk
f"SELECT `id`, `key_` FROM `{service_name}` WHERE `kid`=%s AND `key_`!=%s",
(kid, "0" * 32),
)
cek = cursor.fetchone()
if cek:
return cek["key_"]
return None
finally:
cursor.close()

View File

@@ -19,22 +19,30 @@ class SQLite(Vault):
self.conn_factory = ConnectionFactory(self.path)
def get_key(self, kid: Union[UUID, str], service: str) -> Optional[str]:
if not self.has_table(service):
# no table, no key, simple
return None
if isinstance(kid, UUID):
kid = kid.hex
conn = self.conn_factory.get()
cursor = conn.cursor()
# Try both the original service name and lowercase version to handle case sensitivity issues
service_variants = [service]
if service != service.lower():
service_variants.append(service.lower())
if service != service.upper():
service_variants.append(service.upper())
try:
cursor.execute(f"SELECT `id`, `key_` FROM `{service}` WHERE `kid`=? AND `key_`!=?", (kid, "0" * 32))
cek = cursor.fetchone()
if not cek:
return None
return cek[1]
for service_name in service_variants:
if not self.has_table(service_name):
continue
cursor.execute(f"SELECT `id`, `key_` FROM `{service_name}` WHERE `kid`=? AND `key_`!=?", (kid, "0" * 32))
cek = cursor.fetchone()
if cek:
return cek[1]
return None
finally:
cursor.close()

2
uv.lock generated
View File

@@ -1499,7 +1499,7 @@ wheels = [
[[package]]
name = "unshackle"
version = "1.4.4"
version = "1.4.6"
source = { editable = "." }
dependencies = [
{ name = "appdirs" },