noham/N_m3u8DL-RE
1
0
Fork 0
mirror of https://github.com/NohamR/N_m3u8DL-RE.git synced 2025-05-24 14:21:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

init

Browse Source
This commit is contained in:
√(noham)² 2025-02-01 23:47:52 +01:00
parent 1b498cfe8d
commit 9a87474411
105 changed files with 14268 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitattributes vendored Normal file
View File

@ -0,0 +1,2 @@
# Auto detect text files and perform LF normalization
* text=auto

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2022 nilaoda
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

252
README.md Normal file
View File

@ -0,0 +1,252 @@
# N_m3u8DL-RE
跨平台的DASH/HLS/MSS下载工具。支持点播、直播(DASH/HLS)。
[![img](https://img.shields.io/github/stars/nilaoda/N_m3u8DL-RE?label=%E7%82%B9%E8%B5%9E)](https://github.com/nilaoda/N_m3u8DL-RE) [![img](https://img.shields.io/github/last-commit/nilaoda/N_m3u8DL-RE?label=%E6%9C%80%E8%BF%91%E6%8F%90%E4%BA%A4)](https://github.com/nilaoda/N_m3u8DL-RE) [![img](https://img.shields.io/github/release/nilaoda/N_m3u8DL-RE?label=%E6%9C%80%E6%96%B0%E7%89%88%E6%9C%AC)](https://github.com/nilaoda/N_m3u8DL-RE/releases) [![img](https://img.shields.io/github/license/nilaoda/N_m3u8DL-RE?label=%E8%AE%B8%E5%8F%AF%E8%AF%81)](https://github.com/nilaoda/N_m3u8DL-RE) [![img](https://img.shields.io/github/downloads/nilaoda/N_m3u8DL-RE/total?label=%E4%B8%8B%E8%BD%BD%E9%87%8F)](https://github.com/nilaoda/N_m3u8DL-RE/releases)
遇到 BUG 请首先确认软件是否为最新版本（如果是 Release 版本，建议到 [Actions](https://github.com/nilaoda/N_m3u8DL-RE/actions) 页面下载最新自动构建版本后查看问题是否已经被修复），如果确认版本最新且问题依旧存在，可以到 [Issues](https://github.com/nilaoda/N_m3u8DL-RE/issues) 中查找是否有人遇到过相关问题，没有的话再进行询问。
---
版本较低的Windows系统自带的终端可能不支持本程序，替代方案：在 [cmder](https://github.com/cmderdev/cmder) 中运行。
Arch Linux 可以从 AUR 获取：[n-m3u8dl-re-bin](https://aur.archlinux.org/packages/n-m3u8dl-re-bin)、[n-m3u8dl-re-git](https://aur.archlinux.org/packages/n-m3u8dl-re-git)
```bash
# Arch Linux 及其衍生版安装 N_m3u8DL-RE 发行版 (该源非本人维护)
yay -Syu n-m3u8dl-re-bin
# Arch Linux 及其衍生版安装 N_m3u8DL-RE 开发版 (该源非本人维护)
yay -Syu n-m3u8dl-re-git
```
---
# 命令行参数
```
Description:
N_m3u8DL-RE (Beta version) 20241201
Usage:
N_m3u8DL-RE <input> [options]
Arguments:
<input> 链接或文件
Options:
--tmp-dir <tmp-dir> 设置临时文件存储目录
--save-dir <save-dir> 设置输出目录
--save-name <save-name> 设置保存文件名
--base-url <base-url> 设置BaseURL
--thread-count <number> 设置下载线程数 [default: 本机CPU线程数]
--download-retry-count <number> 每个分片下载异常时的重试次数 [default: 3]
--http-request-timeout <seconds> HTTP请求的超时时间(秒) [default: 100]
--force-ansi-console 强制认定终端为支持ANSI且可交互的终端
--no-ansi-color 去除ANSI颜色
--auto-select 自动选择所有类型的最佳轨道 [default: False]
--skip-merge 跳过合并分片 [default: False]
--skip-download 跳过下载 [default: False]
--check-segments-count 检测实际下载的分片数量和预期数量是否匹配 [default: True]
--binary-merge 二进制合并 [default: False]
--use-ffmpeg-concat-demuxer 使用 ffmpeg 合并时，使用 concat 分离器而非 concat 协议 [default: False]
--del-after-done 完成后删除临时文件 [default: True]
--no-date-info 混流时不写入日期信息 [default: False]
--no-log 关闭日志文件输出 [default: False]
--write-meta-json 解析后的信息是否输出json文件 [default: True]
--append-url-params 将输入Url的Params添加至分片, 对某些网站很有用, 例如 kakao.com [default: False]
-mt, --concurrent-download 并发下载已选择的音频、视频和字幕 [default: False]
-H, --header <header> 为HTTP请求设置特定的请求头, 例如:
-H "Cookie: mycookie" -H "User-Agent: iOS"
--sub-only 只选取字幕轨道 [default: False]
--sub-format <SRT|VTT> 字幕输出类型 [default: SRT]
--auto-subtitle-fix 自动修正字幕 [default: True]
--ffmpeg-binary-path <PATH> ffmpeg可执行程序全路径, 例如 C:\Tools\ffmpeg.exe
--log-level <DEBUG|ERROR|INFO|OFF|WARN> 设置日志级别 [default: INFO]
--ui-language <en-US|zh-CN|zh-TW> 设置UI语言
--urlprocessor-args <urlprocessor-args> 此字符串将直接传递给URL Processor
--key <key> 设置解密密钥, 程序调用mp4decrpyt/shaka-packager/ffmpeg进行解密. 格式:
--key KID1:KEY1 --key KID2:KEY2
对于KEY相同的情况可以直接输入 --key KEY
--key-text-file <key-text-file> 设置密钥文件,程序将从文件中按KID搜寻KEY以解密.(不建议使用特大文件)
--decryption-engine <FFMPEG|MP4DECRYPT|SHAKA_PACKAGER> 设置解密时使用的第三方程序 [default: MP4DECRYPT]
--decryption-binary-path <PATH> MP4解密所用工具的全路径, 例如 C:\Tools\mp4decrypt.exe
--mp4-real-time-decryption 实时解密MP4分片 [default: False]
-R, --max-speed <SPEED> 设置限速，单位支持 Mbps 或 Kbps，如：15M 100K
-M, --mux-after-done <OPTIONS> 所有工作完成时尝试混流分离的音视频. 输入 "--morehelp mux-after-done" 以查看详细信息
--custom-hls-method <METHOD> 指定HLS加密方式 (AES_128|AES_128_ECB|CENC|CHACHA20|NONE|SAMPLE_AES|SAMPLE_AES_CTR|UNKNOWN)
--custom-hls-key <FILE|HEX|BASE64> 指定HLS解密KEY. 可以是文件, HEX或Base64
--custom-hls-iv <FILE|HEX|BASE64> 指定HLS解密IV. 可以是文件, HEX或Base64
--use-system-proxy 使用系统默认代理 [default: True]
--custom-proxy <URL> 设置请求代理, 如 http://127.0.0.1:8888
--custom-range <RANGE> 仅下载部分分片. 输入 "--morehelp custom-range" 以查看详细信息
--task-start-at <yyyyMMddHHmmss> 在此时间之前不会开始执行任务
--live-perform-as-vod 以点播方式下载直播流 [default: False]
--live-real-time-merge 录制直播时实时合并 [default: False]
--live-keep-segments 录制直播并开启实时合并时依然保留分片 [default: True]
--live-pipe-mux 录制直播并开启实时合并时通过管道+ffmpeg实时混流到TS文件 [default: False]
--live-fix-vtt-by-audio 通过读取音频文件的起始时间修正VTT字幕 [default: False]
--live-record-limit <HH:mm:ss> 录制直播时的录制时长限制
--live-wait-time <SEC> 手动设置直播列表刷新间隔
--live-take-count <NUM> 手动设置录制直播时首次获取分片的数量 [default: 16]
--mux-import <OPTIONS> 混流时引入外部媒体文件. 输入 "--morehelp mux-import" 以查看详细信息
-sv, --select-video <OPTIONS> 通过正则表达式选择符合要求的视频流. 输入 "--morehelp select-video" 以查看详细信息
-sa, --select-audio <OPTIONS> 通过正则表达式选择符合要求的音频流. 输入 "--morehelp select-audio" 以查看详细信息
-ss, --select-subtitle <OPTIONS> 通过正则表达式选择符合要求的字幕流. 输入 "--morehelp select-subtitle" 以查看详细信息
-dv, --drop-video <OPTIONS> 通过正则表达式去除符合要求的视频流.
-da, --drop-audio <OPTIONS> 通过正则表达式去除符合要求的音频流.
-ds, --drop-subtitle <OPTIONS> 通过正则表达式去除符合要求的字幕流.
--ad-keyword <REG> 设置广告分片的URL关键字(正则表达式)
--disable-update-check 禁用版本更新检测 [default: False]
--allow-hls-multi-ext-map 允许HLS中的多个#EXT-X-MAP(实验性) [default: False]
--morehelp <OPTION> 查看某个选项的详细帮助信息
--version Show version information
-?, -h, --help Show help and usage information
```
<details>
<summary>点击查看More Help</summary>
```
More Help:
--mux-after-done
所有工作完成时尝试混流分离的音视频. 你能够以:分隔形式指定如下参数:
* format=FORMAT: 指定混流容器 mkv, mp4
* muxer=MUXER: 指定混流程序 ffmpeg, mkvmerge (默认: ffmpeg)
* bin_path=PATH: 指定程序路径 (默认: 自动寻找)
* skip_sub=BOOL: 是否忽略字幕文件 (默认: false)
* keep=BOOL: 混流完成是否保留文件 true, false (默认: false)
例如:
# 混流为mp4容器
-M format=mp4
# 使用mkvmerge, 自动寻找程序
-M format=mkv:muxer=mkvmerge
# 使用mkvmerge, 自定义程序路径
-M format=mkv:muxer=mkvmerge:bin_path="C\:\Program Files\MKVToolNix\mkvmerge.exe"
```
```
More Help:
--mux-import
混流时引入外部媒体文件. 你能够以:分隔形式指定如下参数:
* path=PATH: 指定媒体文件路径
* lang=CODE: 指定媒体文件语言代码 (非必须)
* name=NAME: 指定媒体文件描述信息 (非必须)
例如:
# 引入外部字幕
--mux-import path=zh-Hans.srt:lang=chi:name="中文 (简体)"
# 引入外部音轨+字幕
--mux-import path="D\:\media\atmos.m4a":lang=eng:name="English Description Audio" --mux-import path="D\:\media\eng.vtt":lang=eng:name="English (Description)"
```
```
More Help:
--select-video
通过正则表达式选择符合要求的视频流. 你能够以:分隔形式指定如下参数:
id=REGEX:lang=REGEX:name=REGEX:codecs=REGEX:res=REGEX:frame=REGEX
segsMin=number:segsMax=number:ch=REGEX:range=REGEX:url=REGEX
plistDurMin=hms:plistDurMax=hms:for=FOR
* for=FOR: 选择方式. best[number], worst[number], all (默认: best)
例如:
# 选择最佳视频
-sv best
# 选择4K+HEVC视频
-sv res="3840*":codecs=hvc1:for=best
# 选择长度大于1小时20分钟30秒的视频
-sv plistDurMin="1h20m30s":for=best
```
```
More Help:
--select-audio
通过正则表达式选择符合要求的音频流. 参考 --select-video
例如:
# 选择所有音频
-sa all
# 选择最佳英语音轨
-sa lang=en:for=best
# 选择最佳的2条英语(或日语)音轨
-sa lang="ja|en":for=best2
```
```
More Help:
--select-subtitle
通过正则表达式选择符合要求的字幕流. 参考 --select-video
例如:
# 选择所有字幕
-ss all
# 选择所有带有"中文"的字幕
-ss name="中文":for=all
```
```
More Help:
--custom-range
下载点播内容时, 仅下载部分分片.
例如:
# 下载[0,10]共11个分片
--custom-range 0-10
# 下载从序号10开始的后续分片
--custom-range 10-
# 下载前100个分片
--custom-range -99
# 下载第5分钟到20分钟的内容
--custom-range 05:00-20:00
```
</details>
# 运行截图
## 点播
![RE1](img/RE.gif)
还可以并行下载+自动混流
![RE2](img/RE2.gif)
## 直播
录制TS直播源：
[click to show gif](http://pan.iqiyi.com/file/paopao/W0LfmaMRvuA--uCdOpZ1cldM5JCVhMfIm7KFqr4oKCz80jLn0bBb-9PWmeCFZ-qHpAaQydQ1zk-CHYT_UbRLtw.gif)
录制MPD直播源：
[click to show gif](http://pan.iqiyi.com/file/paopao/nmAV5MOh0yIyHhnxdgM_6th_p2nqrFsM4k-o3cUPwUa8Eh8QOU4uyPkLa_BlBrMa3GBnKWSk8rOaUwbsjKN14g.gif)
录制过程中，借助ffmpeg完成对音视频的实时混流
```
ffmpeg -readrate 1 -i 2022-09-21_19-54-42_V.mp4 -i 2022-09-21_19-54-42_V.chi.m4a -c copy 2022-09-21_19-54-42_V.ts
```
在新版本(>=v0.1.5)中，可以尝试开启 `live-pipe-mux` 来代替以上命令
**特别注意：如果网络环境不够稳定，请不要开启 `live-pipe-mux`。管道内数据读取由 ffmpeg 负责，在某些环境下容易丢失直播数据**
在新版本(>=v0.1.8)中，能够通过设置环境变量 `RE_LIVE_PIPE_OPTIONS` 来改变 `live-pipe-mux` 时 ffmpeg 的某些选项： https://github.com/nilaoda/N_m3u8DL-RE/issues/162#issuecomment-1592462532
## 赞助
<a href="https://www.buymeacoffee.com/nilaoda" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="41" width="174"></a>

39
TestStreams.md Normal file
View File

@ -0,0 +1,39 @@
# Test Streams
* https://vod-ftc-eu-west-1.media.dssott.com/ps01/disney/29a73209-b706-4f21-8384-acddccb154d2/ctr-all-6cf6fec6-94dc-4f8b-ae67-5ada60ee1e83-42bd5bca-f9a8-4299-83e3-0fb2b4ec0a62.m3u8 (迪士尼)
* https://play.itunes.apple.com/WebObjects/MZPlay.woa/hls/subscription/playlist.m3u8?cc=US&svcId=tvs.vds.4105&a=1580273278&isExternal=true&brandId=tvs.sbd.4000&id=337246031&l=en-US&aec=UHD&xtrick=true&webbrowser=true (啥都有)
* https://media.axprod.net/TestVectors/v7-Clear/Manifest_1080p.mpd (多音轨多字幕)
* https://cmafref.akamaized.net/cmaf/live-ull/2006350/akambr/out.mpd (直播)
* http://playertest.longtailvideo.com/adaptive/oceans_aes/oceans_aes.m3u8
* https://vod.sdn.wavve.com/hls/S01/S01_E461382925.1/1/5000/chunklist.m3u8
* https://bitmovin-a.akamaihd.net/content/art-of-motion_drm/mpds/11331.mpd
* http://tv-live.ynkmit.com/tv/anning.m3u8?txSecret=7528f35fb4b62bd24d55b891899db68f&txTime=632C8680 (直播)
* https://rest-as.ott.kaltura.com/api_v3/service/assetFile/action/playManifest/partnerId/147/assetId/1304099/assetType/media/assetFileId/16136929/contextType/PLAYBACK/isAltUrl/False/ks/djJ8MTQ3fMusTFH6PCZpcrfKLQwI-pPm9ex6b6r49wioe32WH2udXeM4reyWIkSDpi7HhvhxBHAHAKiHrcnkmIJQpyAt4MuDBG0ywGQ-jOeqQFcTRQ8BGJGw6g-smSBLwSbo4CCx9M9vWNJX3GkOfhoMAY4yRU-ur3okHiVq1mUJ82XBd_iVqLuzodnc9sJEtcHH0zc5CoPiTq2xor-dq3yDURnZm3isfSN3t9uLIJEW09oE-SJ84DM5GUuFUdbnIV8bdcWUsPicUg-Top1G2D3WcWXq4EvPnwvD8jrC_vsiOpLHf5akAwtdGsJ6__cXUmT7a-QlfjdvaZ5T8UhDLnttHmsxYs2E5c0lh4uOvvJou8dD8iYxUexlPI2j4QUkBRxqOEVLSNV3Y82-5TTRqgnK_uGYXHwk7EAmDws7hbLj2-DJ1heXDcye3OJYdunJgAS-9ma5zmQQNiY_HYh6wj2N1HpCTNAtWWga6R9fC0VgBTZbidW-YwMSGzIvMQfIfWKe15X7Oc_hCs-zGfW9XeRJZrutcWKK_D_HlzpQVBF2vIF3XgaI/a.mpd
* https://dash.akamaized.net/dash264/TestCases/2c/qualcomm/1/MultiResMPEG2.mpd
* https://cmaf.lln.latam.hbomaxcdn.com/videos/GYPGKMQjoDkVLBQEAAAAo/1/1b5ad5/1_single_J8sExA_1080hi.mpd
* https://livesim.dashif.org/dash/vod/testpic_2s/multi_subs.mpd (ttml + mp4)
* http://media.axprod.net/TestVectors/v6-Clear/Manifest_1080p.mpd (vtt + mp4)
* https://livesim.dashif.org/dash/vod/testpic_2s/xml_subs.mpd (ttml)
* https://storage.googleapis.com/shaka-demo-assets/angel-one-hls/hls.m3u8 (HLS vtt)
* https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_adv_example_hevc/master.m3u8 (高级HLS fMP4+VTT)
* https://events-delivery.apple.com/0205eyyhwbbqexozkwmgccegwnjyrktg/m3u8/vod_index-dpyfrsVksFWjneFiptbXnAMYBtGYbXeZ.m3u8 (高级HLS fMP4+VTT)
* https://apionvod5.seezntv.com/ktmain1/cold/CP/55521/202207/media/MIAM61RPSGL150000100_DRM/MIAM61RPSGL150000100_H.m3u8?sid=0000000F50000040000A700000020000
* https://ewcdn12.nowe.com/session/16-5-72579e3-2103014898783810281/Content/DASH_VOS3/VOD/6908/19585/d2afa5fe-e9c8-40f0-8d18-648aaaf292b6/f677841a-9d8f-2ff5-3517-674ba49ef192/manifest.mpd?token=894db5d69931835f82dd8e393974ef9f_1658146180
* https://ols-ww100-cp.akamaized.net/manifest/master/06ee6f68-ee80-11ea-9bc5-02b68fb543c4/65794a72596d6c30496a6f7a4e6a67324e4441774d444173496e42735958526d62334a74496a6f695a47567a6133527663434973496d526c646d6c6a5a565235634755694f694a335a5749694c434a746232526c62434936496e6470626d527664334d694c434a7663315235634755694f694a6a61484a76625755694c434a7663794936496a45774d6934774c6a41694c434a68634841694f69497a4c6a416966513d3d/dash.mpd?cpatoken=exp=1658223027~acl=/manifest/master/06ee6f68-ee80-11ea-9bc5-02b68fb543c4/*~hmac=644c608aac361f688e9b24b0f345c801d0f2d335819431d1873ff7aeac46d6b2&access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJkZXZpY2VfaWQiOm51bGwsIndhdGNoX3R5cGUiOiJQUkVNSVVNIiwicHJvZ3JhbV9pZCI6ImUwMWRmYjAyLTM1YmItMTFlOS1hNDI3LTA2YTA0MTdjMWQxZSIsImFkX3RhZyI6ZmFsc2UsInBhcmVudF9wcm9ncmFtX2lkIjoiZmJmMDc2MDYtMzNmYi0xMWU5LWE0MjctMDZhMDQxN2MxZDFlIiwiY2xpZW50X2lkIjoiNGQ3MDViZTQtYTQ5ZS0xMWVhLWJiMzctMDI0MmFjMTMwMDAyIiwidmlkZW9fdHlwZSI6InZvZCIsImdyYW50X3R5cGUiOiJwbGF5X3ZpZGVvIiwidXNlcl9pZCI6ImFhNTMxZWQ2LWM2NTMtNDliYS04NGI1LWFkZDRmNGIzNGMyNyIsImN1cnJlbnRfc2Vjb25kIjowLCJyZXBvcnRfaWQiOiJOU1RHIiwic2NvcGUiOlsicHVibGljOi4qIiwibWU6LioiXSwiZXhwIjoxNjU4Mzk1ODI2LCJkZXRlY3Rpb25faWQiOm51bGwsInZpZGVvX2lkIjoiODc0Yjk0ZDItNzZiYi00YzliLTgzODQtNzJlMTA0NWVjOGMxIiwiaXNzIjoiQXNpYXBsYXktT0F1dGgtU2VydmVyIiwiaWF0IjoxNjU4MTM2NjI2LCJ0ZXJyaXRvcnkiOiJUVyJ9.1juciYIyMNzykXKu-nGLR_cYWvPMEAE9ub-ny7RzFnM
* https://a38avoddashs3ww-a.akamaihd.net/ondemand/iad_2/8e91/f2f2/ec5a/430f-bd7a-0779f4a0189d/685cda75-609c-41c1-86bb-688f4cdb5521_corrected.mpd
* https://dcs-vod.mp.lura.live/vod/p/session/manifest.mpd?i=i177610817-nb45239a2-e962-4137-bc70-1790359619e6
* https://theater.kktv.com.tw/98/04000198010001_584b26392f7f7f11fc62299214a55fb7/16113081449d8d5e9960_sub_dash.mpd (MPD+VTT)
* https://vsl.play.kakao.com/vod/rvty90n7btua6u9oebr97i8zl/dash/vhs/cenc/adaptive.mpd?e=1658297362&p=71&h=53766bdde112d59da2b2514e8ab41e81 (需要补params)
* https://a38avoddashs3ww-a.akamaihd.net/ondemand/iad_2/8e91/f2f2/ec5a/430f-bd7a-0779f4a0189d/685cda75-609c-41c1-86bb-688f4cdb5521_corrected.mpd
* http://ht.grelighting.cn/m3u8/OEtYNVNjMFF5N1g2VzNkZ2lwbWEvd1ZtTGJ0dlZXOEk=.m3u8 (特殊的图片伪装)
* https://ali6.a.yximgs.com/udata/music/music_295e59bdb3084def8158873ad6f5c8250.jpg (PNG图片伪装)
* https://vod.cds.nowonline.com.br/Content/dsc/VOD/movie/df/83/4dd69316-2022-45c0-a4f7-c35cad87df83/manifest.mpd (TTML+PNG)
* https://m-3s3.ll.smooth.row.aiv-cdn.net/iad_2/b473/00fa/20a4/4f93-bc40-1f4462cbba74/5f923727-474e-4788-ab2f-6ebc512a5be3.ism/manifest (ISM test1)
* https://ec21-waw1.waw2.cache.orange.pl/canal/v/canal/vod/store01/CSM_21006719/_/hd4-hssdrm02.ism/Manifest
* http://playready.directtaps.net/smoothstreaming/SSWSS720H264/SuperSpeedway_720.ism/Manifest
* http://amssamples.streaming.mediaservices.windows.net/683f7e47-bd83-4427-b0a3-26a6c4547782/BigBuckBunny.ism/manifest
* https://azclwds01.akamaized.net/4e8f6858-5d05-4e28-83ab-48c7a2b259e1/XVuosg_tab_hd.ism/Manifest (`e5114f87b3e54b9faca331e6e6d646a6:55c5d9f1cedfd018b75623f2565a1d29`)
* https://cd-stream-live.telenorcdn.net/cdgo/cdgo_tv2hd_dk_live/cdgo_tv2hd_dk_live.isml/Manifest (`a1447cecef3a4e839d59c1f994eaf2c6:86a376288496a0c32e1178e42139ce66`, **LIVE**)
* https://linear017-gb-hss1-prd-ak.cdn.skycdp.com/100e/Content/SS_002_sd/Live/channel(skycinemaanimation).isml/Manifest_sd (`8f2b488c1214a4085c34968f8505910a:fe607f4050b2b217b0eee7906d196cca`, **LIVE**, **NOT** Supported Yet)
* https://avpanta-cdn.pantaflix.com/cl/prod/content/CO2568_CO2568_Intro_e3e4814b_V1/C3CMm43Aucy-TVOD-dash/main_feature_DASH.ism/Manifest (`0751e816fce5433aaab2b775ef5c2131:f771897857b42d31e64d10d551376dcf`)
* http://ss-cdn1.blim.com/PAID0000000000756826/ss/66aa2b5c2bafc73753fd6ad23db679330e3a35ba32931990125d5b41092e4291/ss_66aa2b5c2bafc73753fd6ad23db679330e3a35ba32931990125d5b41092e4291.ism/Manifest (`b6e16839eebd4ff6ab768d482d8d2b6a:ad6c675e0810741538f7f2f0b4099d9e`, **Invalid** Language)

BIN
img/RE.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

BIN
img/RE2.gif Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

34
src/N_m3u8DL-RE.Common/Entity/EncryptInfo.cs Normal file
View File

@ -0,0 +1,34 @@
using N_m3u8DL_RE.Common.Enum;
namespace N_m3u8DL_RE.Common.Entity;
public class EncryptInfo
{
/// <summary>
/// 加密方式，默认无加密
/// </summary>
public EncryptMethod Method { get; set; } = EncryptMethod.NONE;
public byte[]? Key { get; set; }
public byte[]? IV { get; set; }
public EncryptInfo() { }
/// <summary>
/// 创建EncryptInfo并尝试自动解析Method
/// </summary>
/// <param name="method"></param>
public EncryptInfo(string method)
{
Method = ParseMethod(method);
}
public static EncryptMethod ParseMethod(string? method)
{
if (method != null && System.Enum.TryParse(method.Replace("-", "_"), out EncryptMethod m))
{
return m;
}
return EncryptMethod.UNKNOWN;
}
}

18
src/N_m3u8DL-RE.Common/Entity/MSSData.cs Normal file
View File

@ -0,0 +1,18 @@
namespace N_m3u8DL_RE.Common.Entity;
public class MSSData
{
public string FourCC { get; set; } = "";
public string CodecPrivateData { get; set; } = "";
public string Type { get; set; } = "";
public int Timesacle { get; set; }
public int SamplingRate { get; set; }
public int Channels { get; set; }
public int BitsPerSample { get; set; }
public int NalUnitLengthField { get; set; }
public long Duration { get; set; }
public bool IsProtection { get; set; } = false;
public string ProtectionSystemID { get; set; } = "";
public string ProtectionData { get; set; } = "";
}

7
src/N_m3u8DL-RE.Common/Entity/MediaPart.cs Normal file
View File

@ -0,0 +1,7 @@
namespace N_m3u8DL_RE.Common.Entity;
// 主要处理 EXT-X-DISCONTINUITY
public class MediaPart
{
public List<MediaSegment> MediaSegments { get; set; } = [];
}

40
src/N_m3u8DL-RE.Common/Entity/MediaSegment.cs Normal file
View File

@ -0,0 +1,40 @@
using N_m3u8DL_RE.Common.Enum;
namespace N_m3u8DL_RE.Common.Entity;
public class MediaSegment
{
public long Index { get; set; }
public double Duration { get; set; }
public string? Title { get; set; }
public DateTime? DateTime { get; set; }
public long? StartRange { get; set; }
public long? StopRange => (StartRange != null && ExpectLength != null) ? StartRange + ExpectLength - 1 : null;
public long? ExpectLength { get; set; }
public EncryptInfo EncryptInfo { get; set; } = new();
public bool IsEncrypted => EncryptInfo.Method != EncryptMethod.NONE;
public string Url { get; set; } = string.Empty;
public string? NameFromVar { get; set; } // MPD分段文件名
public override bool Equals(object? obj)
{
return obj is MediaSegment segment &&
Index == segment.Index &&
Math.Abs(Duration - segment.Duration) < 0.001 &&
Title == segment.Title &&
StartRange == segment.StartRange &&
StopRange == segment.StopRange &&
ExpectLength == segment.ExpectLength &&
Url == segment.Url;
}
public override int GetHashCode()
{
return HashCode.Combine(Index, Duration, Title, StartRange, StopRange, ExpectLength, Url);
}
}

20
src/N_m3u8DL-RE.Common/Entity/Playlist.cs Normal file
View File

@ -0,0 +1,20 @@
namespace N_m3u8DL_RE.Common.Entity;
public class Playlist
{
// 对应Url信息
public string Url { get; set; } = string.Empty;
// 是否直播
public bool IsLive { get; set; } = false;
// 直播刷新间隔毫秒（默认15秒）
public double RefreshIntervalMs { get; set; } = 15000;
// 所有分片时长总和
public double TotalDuration => MediaParts.Sum(x => x.MediaSegments.Sum(m => m.Duration));
// 所有分片中最长时长
public double? TargetDuration { get; set; }
// INIT信息
public MediaSegment? MediaInit { get; set; }
// 分片信息
public List<MediaPart> MediaParts { get; set; } = [];
}

182
src/N_m3u8DL-RE.Common/Entity/StreamSpec.cs Normal file
View File

@ -0,0 +1,182 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Util;
using Spectre.Console;
namespace N_m3u8DL_RE.Common.Entity;
public class StreamSpec
{
public MediaType? MediaType { get; set; }
public string? GroupId { get; set; }
public string? Language { get; set; }
public string? Name { get; set; }
public Choise? Default { get; set; }
// 由于用户选择 被跳过的分片总时长
public double? SkippedDuration { get; set; }
// MSS信息
public MSSData? MSSData { get; set; }
// 基本信息
public int? Bandwidth { get; set; }
public string? Codecs { get; set; }
public string? Resolution { get; set; }
public double? FrameRate { get; set; }
public string? Channels { get; set; }
public string? Extension { get; set; }
// Dash
public RoleType? Role { get; set; }
// 补充信息-色域
public string? VideoRange { get; set; }
// 补充信息-特征
public string? Characteristics { get; set; }
// 发布时间（仅MPD需要）
public DateTime? PublishTime { get; set; }
// 外部轨道GroupId (后续寻找对应轨道信息)
public string? AudioId { get; set; }
public string? VideoId { get; set; }
public string? SubtitleId { get; set; }
public string? PeriodId { get; set; }
/// <summary>
/// URL
/// </summary>
public string Url { get; set; } = string.Empty;
/// <summary>
/// 原始URL
/// </summary>
public string OriginalUrl { get; set; } = string.Empty;
public Playlist? Playlist { get; set; }
public int SegmentsCount
{
get
{
return Playlist != null ? Playlist.MediaParts.Sum(x => x.MediaSegments.Count) : 0;
}
}
public string ToShortString()
{
var prefixStr = "";
var returnStr = "";
var encStr = string.Empty;
if (MediaType == Enum.MediaType.AUDIO)
{
prefixStr = $"[deepskyblue3]Aud[/] {encStr}";
var d = $"{GroupId} | {(Bandwidth != null ? (Bandwidth / 1000) + " Kbps" : "")} | {Name} | {Codecs} | {Language} | {(Channels != null ? Channels + "CH" : "")} | {Role}";
returnStr = d.EscapeMarkup();
}
else if (MediaType == Enum.MediaType.SUBTITLES)
{
prefixStr = $"[deepskyblue3_1]Sub[/] {encStr}";
var d = $"{GroupId} | {Language} | {Name} | {Codecs} | {Role}";
returnStr = d.EscapeMarkup();
}
else
{
prefixStr = $"[aqua]Vid[/] {encStr}";
var d = $"{Resolution} | {Bandwidth / 1000} Kbps | {GroupId} | {FrameRate} | {Codecs} | {VideoRange} | {Role}";
returnStr = d.EscapeMarkup();
}
returnStr = prefixStr + returnStr.Trim().Trim('|').Trim();
while (returnStr.Contains("| |"))
{
returnStr = returnStr.Replace("| |", "|");
}
return returnStr.TrimEnd().TrimEnd('|').TrimEnd();
}
public string ToShortShortString()
{
var prefixStr = "";
var returnStr = "";
var encStr = string.Empty;
if (MediaType == Enum.MediaType.AUDIO)
{
prefixStr = $"[deepskyblue3]Aud[/] {encStr}";
var d = $"{(Bandwidth != null ? (Bandwidth / 1000) + " Kbps" : "")} | {Name} | {Language} | {(Channels != null ? Channels + "CH" : "")} | {Role}";
returnStr = d.EscapeMarkup();
}
else if (MediaType == Enum.MediaType.SUBTITLES)
{
prefixStr = $"[deepskyblue3_1]Sub[/] {encStr}";
var d = $"{Language} | {Name} | {Codecs} | {Role}";
returnStr = d.EscapeMarkup();
}
else
{
prefixStr = $"[aqua]Vid[/] {encStr}";
var d = $"{Resolution} | {Bandwidth / 1000} Kbps | {FrameRate} | {VideoRange} | {Role}";
returnStr = d.EscapeMarkup();
}
returnStr = prefixStr + returnStr.Trim().Trim('|').Trim();
while (returnStr.Contains("| |"))
{
returnStr = returnStr.Replace("| |", "|");
}
return returnStr.TrimEnd().TrimEnd('|').TrimEnd();
}
public override string ToString()
{
var prefixStr = "";
var returnStr = "";
var encStr = string.Empty;
var segmentsCountStr = SegmentsCount == 0 ? "" : (SegmentsCount > 1 ? $"{SegmentsCount} Segments" : $"{SegmentsCount} Segment");
// 增加加密标志
if (Playlist != null && Playlist.MediaParts.Any(m => m.MediaSegments.Any(s => s.EncryptInfo.Method != EncryptMethod.NONE)))
{
var ms = Playlist.MediaParts.SelectMany(m => m.MediaSegments.Select(s => s.EncryptInfo.Method)).Where(e => e != EncryptMethod.NONE).Distinct();
encStr = $"[red]*{string.Join(",", ms).EscapeMarkup()}[/] ";
}
if (MediaType == Enum.MediaType.AUDIO)
{
prefixStr = $"[deepskyblue3]Aud[/] {encStr}";
var d = $"{GroupId} | {(Bandwidth != null ? (Bandwidth / 1000) + " Kbps" : "")} | {Name} | {Codecs} | {Language} | {(Channels != null ? Channels + "CH" : "")} | {segmentsCountStr} | {Role}";
returnStr = d.EscapeMarkup();
}
else if (MediaType == Enum.MediaType.SUBTITLES)
{
prefixStr = $"[deepskyblue3_1]Sub[/] {encStr}";
var d = $"{GroupId} | {Language} | {Name} | {Codecs} | {Characteristics} | {segmentsCountStr} | {Role}";
returnStr = d.EscapeMarkup();
}
else
{
prefixStr = $"[aqua]Vid[/] {encStr}";
var d = $"{Resolution} | {Bandwidth / 1000} Kbps | {GroupId} | {FrameRate} | {Codecs} | {VideoRange} | {segmentsCountStr} | {Role}";
returnStr = d.EscapeMarkup();
}
returnStr = prefixStr + returnStr.Trim().Trim('|').Trim();
while (returnStr.Contains("| |"))
{
returnStr = returnStr.Replace("| |", "|");
}
// 计算时长
if (Playlist != null)
{
var total = Playlist.TotalDuration;
returnStr += " | ~" + GlobalUtil.FormatTime((int)total);
}
return returnStr.TrimEnd().TrimEnd('|').TrimEnd();
}
}

23
src/N_m3u8DL-RE.Common/Entity/SubCue.cs Normal file
View File

@ -0,0 +1,23 @@
namespace N_m3u8DL_RE.Common.Entity;
public class SubCue
{
public TimeSpan StartTime { get; set; }
public TimeSpan EndTime { get; set; }
public required string Payload { get; set; }
public required string Settings { get; set; }
public override bool Equals(object? obj)
{
return obj is SubCue cue &&
StartTime.Equals(cue.StartTime) &&
EndTime.Equals(cue.EndTime) &&
Payload == cue.Payload &&
Settings == cue.Settings;
}
public override int GetHashCode()
{
return HashCode.Combine(StartTime, EndTime, Payload, Settings);
}
}

268
src/N_m3u8DL-RE.Common/Entity/WebVttSub.cs Normal file
View File

@ -0,0 +1,268 @@
using System.Text;
using System.Text.RegularExpressions;
namespace N_m3u8DL_RE.Common.Entity;
public partial class WebVttSub
{
[GeneratedRegex("X-TIMESTAMP-MAP.*")]
private static partial Regex TSMapRegex();
[GeneratedRegex("MPEGTS:(\\d+)")]
private static partial Regex TSValueRegex();
[GeneratedRegex("\\s")]
private static partial Regex SplitRegex();
[GeneratedRegex(@"<c\..*?>([\s\S]*?)<\/c>")]
private static partial Regex VttClassRegex();
public List<SubCue> Cues { get; set; } = [];
public long MpegtsTimestamp { get; set; } = 0L;
/// <summary>
/// 从字节数组解析WEBVTT
/// </summary>
/// <param name="textBytes"></param>
/// <returns></returns>
public static WebVttSub Parse(byte[] textBytes, long BaseTimestamp = 0L)
{
return Parse(Encoding.UTF8.GetString(textBytes), BaseTimestamp);
}
/// <summary>
/// 从字节数组解析WEBVTT
/// </summary>
/// <param name="textBytes"></param>
/// <param name="encoding"></param>
/// <returns></returns>
public static WebVttSub Parse(byte[] textBytes, Encoding encoding, long BaseTimestamp = 0L)
{
return Parse(encoding.GetString(textBytes), BaseTimestamp);
}
/// <summary>
/// 从字符串解析WEBVTT
/// </summary>
/// <param name="text"></param>
/// <returns></returns>
public static WebVttSub Parse(string text, long BaseTimestamp = 0L)
{
if (!text.Trim().StartsWith("WEBVTT"))
throw new Exception("Bad vtt!");
text += Environment.NewLine;
var webSub = new WebVttSub();
var needPayload = false;
var timeLine = "";
var regex1 = TSMapRegex();
if (regex1.IsMatch(text))
{
var timestamp = TSValueRegex().Match(regex1.Match(text).Value).Groups[1].Value;
webSub.MpegtsTimestamp = Convert.ToInt64(timestamp);
}
var payloads = new List<string>();
foreach (var line in text.Split('\n'))
{
if (line.Contains(" --> "))
{
needPayload = true;
timeLine = line.Trim();
continue;
}
if (!needPayload) continue;
if (string.IsNullOrEmpty(line.Trim()))
{
var payload = string.Join(Environment.NewLine, payloads);
if (string.IsNullOrEmpty(payload.Trim())) continue; // 没获取到payload 跳过添加
var arr = SplitRegex().Split(timeLine.Replace("-->", "")).Where(s => !string.IsNullOrEmpty(s)).ToList();
var startTime = ConvertToTS(arr[0]);
var endTime = ConvertToTS(arr[1]);
var style = arr.Count > 2 ? string.Join(" ", arr.Skip(2)) : "";
webSub.Cues.Add(new SubCue()
{
StartTime = startTime,
EndTime = endTime,
Payload = RemoveClassTag(string.Join("", payload.Where(c => c != 8203))), // Remove Zero Width Space!
Settings = style
});
payloads.Clear();
needPayload = false;
}
else
{
payloads.Add(line.Trim());
}
}
if (BaseTimestamp == 0) return webSub;
foreach (var item in webSub.Cues)
{
if (item.StartTime.TotalMilliseconds - BaseTimestamp >= 0)
{
item.StartTime = TimeSpan.FromMilliseconds(item.StartTime.TotalMilliseconds - BaseTimestamp);
item.EndTime = TimeSpan.FromMilliseconds(item.EndTime.TotalMilliseconds - BaseTimestamp);
}
else
{
break;
}
}
return webSub;
}
private static string RemoveClassTag(string text)
{
if (VttClassRegex().IsMatch(text))
{
return string.Join(Environment.NewLine, text.Split('\n').Select(line => line.TrimEnd()).Select(line =>
{
return string.Concat(VttClassRegex().Matches(line).Select(x => x.Groups[1].Value + " "));
})).TrimEnd();
}
return text;
}
/// <summary>
/// 从另一个字幕中获取所有Cue，并加载此字幕中，且自动修正偏移
/// </summary>
/// <param name="webSub"></param>
/// <returns></returns>
public WebVttSub AddCuesFromOne(WebVttSub webSub)
{
FixTimestamp(webSub, this.MpegtsTimestamp);
foreach (var item in webSub.Cues)
{
if (this.Cues.Contains(item)) continue;
// 如果相差只有1ms，且payload相同，则拼接
var last = this.Cues.LastOrDefault();
if (last != null && this.Cues.Count > 0 && (item.StartTime - last.EndTime).TotalMilliseconds <= 1 && item.Payload == last.Payload)
{
last.EndTime = item.EndTime;
}
else
{
this.Cues.Add(item);
}
}
return this;
}
private void FixTimestamp(WebVttSub sub, long baseTimestamp)
{
if (sub.MpegtsTimestamp == 0)
{
return;
}
// 确实存在时间轴错误的情况，才修复
if ((this.Cues.Count > 0 && sub.Cues.Count > 0 && sub.Cues.First().StartTime < this.Cues.Last().EndTime && sub.Cues.First().EndTime != this.Cues.Last().EndTime) || this.Cues.Count == 0)
{
// The MPEG2 transport stream clocks (PCR, PTS, DTS) all have units of 1/90000 second
var seconds = (sub.MpegtsTimestamp - baseTimestamp) / 90000;
var offset = TimeSpan.FromSeconds(seconds);
// 当前预添加的字幕的起始时间小于实际上已经走过的时间(如offset已经是100秒，而字幕起始却是2秒)，才修复
if (sub.Cues.Count > 0 && sub.Cues.First().StartTime < offset)
{
foreach (var subCue in sub.Cues)
{
subCue.StartTime += offset;
subCue.EndTime += offset;
}
}
}
}
private IEnumerable<SubCue> GetCues()
{
return this.Cues.Where(c => !string.IsNullOrEmpty(c.Payload));
}
private static TimeSpan ConvertToTS(string str)
{
// 17.0s
if (str.EndsWith('s'))
{
double sec = Convert.ToDouble(str[..^1]);
return TimeSpan.FromSeconds(sec);
}
str = str.Replace(',', '.');
long time = 0;
string[] parts = str.Split('.');
if (parts.Length > 1)
{
time += Convert.ToInt32(parts.Last().PadRight(3, '0'));
str = parts.First();
}
var t = str.Split(':').Reverse().ToList();
for (int i = 0; i < t.Count; i++)
{
time += (long)Math.Pow(60, i) * Convert.ToInt32(t[i]) * 1000;
}
return TimeSpan.FromMilliseconds(time);
}
public override string ToString()
{
var sb = new StringBuilder();
foreach (var c in GetCues()) // 输出时去除空串
{
sb.AppendLine(c.StartTime.ToString(@"hh\:mm\:ss\.fff") + " --> " + c.EndTime.ToString(@"hh\:mm\:ss\.fff") + " " + c.Settings);
sb.AppendLine(c.Payload);
sb.AppendLine();
}
sb.AppendLine();
return sb.ToString();
}
/// <summary>
/// 字幕向前平移指定时间
/// </summary>
/// <param name="time"></param>
public void LeftShiftTime(TimeSpan time)
{
foreach (var cue in this.Cues)
{
if (cue.StartTime.TotalSeconds - time.TotalSeconds > 0) cue.StartTime -= time;
else cue.StartTime = TimeSpan.FromSeconds(0);
if (cue.EndTime.TotalSeconds - time.TotalSeconds > 0) cue.EndTime -= time;
else cue.EndTime = TimeSpan.FromSeconds(0);
}
}
public string ToVtt()
{
return "WEBVTT" + Environment.NewLine + Environment.NewLine + ToString();
}
public string ToSrt()
{
StringBuilder sb = new StringBuilder();
int index = 1;
foreach (var c in GetCues())
{
sb.AppendLine($"{index++}");
sb.AppendLine(c.StartTime.ToString(@"hh\:mm\:ss\,fff") + " --> " + c.EndTime.ToString(@"hh\:mm\:ss\,fff"));
sb.AppendLine(c.Payload);
sb.AppendLine();
}
sb.AppendLine();
var srt = sb.ToString();
if (string.IsNullOrEmpty(srt.Trim()))
{
srt = "1\r\n00:00:00,000 --> 00:00:01,000"; // 空字幕
}
return srt;
}
}

7
src/N_m3u8DL-RE.Common/Enum/Choise.cs Normal file
View File

@ -0,0 +1,7 @@
namespace N_m3u8DL_RE.Common.Enum;
public enum Choise
{
YES = 1,
NO = 0
}

13
src/N_m3u8DL-RE.Common/Enum/EncryptMethod.cs Normal file
View File

@ -0,0 +1,13 @@
namespace N_m3u8DL_RE.Common.Enum;
public enum EncryptMethod
{
NONE,
AES_128,
AES_128_ECB,
SAMPLE_AES,
SAMPLE_AES_CTR,
CENC,
CHACHA20,
UNKNOWN
}

9
src/N_m3u8DL-RE.Common/Enum/ExtractorType.cs Normal file
View File

@ -0,0 +1,9 @@
namespace N_m3u8DL_RE.Common.Enum;
public enum ExtractorType
{
MPEG_DASH,
HLS,
HTTP_LIVE,
MSS
}

9
src/N_m3u8DL-RE.Common/Enum/MediaType.cs Normal file
View File

@ -0,0 +1,9 @@
namespace N_m3u8DL_RE.Common.Enum;
public enum MediaType
{
AUDIO = 0,
VIDEO = 1,
SUBTITLES = 2,
CLOSED_CAPTIONS = 3
}

15
src/N_m3u8DL-RE.Common/Enum/RoleType.cs Normal file
View File

@ -0,0 +1,15 @@
namespace N_m3u8DL_RE.Common.Enum;
public enum RoleType
{
Subtitle = 0,
Main = 1,
Alternate = 2,
Supplementary = 3,
Commentary = 4,
Dub = 5,
Description = 6,
Sign = 7,
Metadata = 8,
ForcedSubtitle = 9
}

21
src/N_m3u8DL-RE.Common/JsonContext/JsonContext.cs Normal file
View File

@ -0,0 +1,21 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using System.Text.Json.Serialization;
namespace N_m3u8DL_RE.Common;
[JsonSourceGenerationOptions(
WriteIndented = true,
PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase,
GenerationMode = JsonSourceGenerationMode.Metadata)]
[JsonSerializable(typeof(MediaType))]
[JsonSerializable(typeof(EncryptMethod))]
[JsonSerializable(typeof(ExtractorType))]
[JsonSerializable(typeof(Choise))]
[JsonSerializable(typeof(StreamSpec))]
[JsonSerializable(typeof(IOrderedEnumerable<StreamSpec>))]
[JsonSerializable(typeof(IEnumerable<MediaSegment>))]
[JsonSerializable(typeof(List<StreamSpec>))]
[JsonSerializable(typeof(List<MediaSegment>))]
[JsonSerializable(typeof(Dictionary<string, string>))]
internal partial class JsonContext : JsonSerializerContext { }

11
src/N_m3u8DL-RE.Common/JsonConverter/BytesBase64Converter.cs Normal file
View File

@ -0,0 +1,11 @@
using System.Text.Json;
using System.Text.Json.Serialization;
namespace N_m3u8DL_RE.Common.JsonConverter;
internal class BytesBase64Converter : JsonConverter<byte[]>
{
public override byte[] Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) => reader.GetBytesFromBase64();
public override void Write(Utf8JsonWriter writer, byte[] value, JsonSerializerOptions options) => writer.WriteStringValue(Convert.ToBase64String(value));
}

99
src/N_m3u8DL-RE.Common/Log/CustomAnsiConsole.cs Normal file
View File

@ -0,0 +1,99 @@
using System.Text;
using System.Text.RegularExpressions;
using Spectre.Console;
namespace N_m3u8DL_RE.Common.Log;
public partial class NonAnsiWriter : TextWriter
{
public override Encoding Encoding => Console.OutputEncoding;
private string? _lastOut = "";
public override void Write(char value)
{
Console.Write(value);
}
public override void Write(string? value)
{
if (_lastOut == value)
{
return;
}
_lastOut = value;
RemoveAnsiEscapeSequences(value);
}
private void RemoveAnsiEscapeSequences(string? input)
{
// Use regular expression to remove ANSI escape sequences
var output = MyRegex().Replace(input ?? "", "");
output = MyRegex1().Replace(output, "");
output = MyRegex2().Replace(output, "");
if (string.IsNullOrWhiteSpace(output))
{
return;
}
Console.Write(output);
}
[GeneratedRegex(@"\x1B\[(\d+;?)+m")]
private static partial Regex MyRegex();
[GeneratedRegex(@"\[\??\d+[AKlh]")]
private static partial Regex MyRegex1();
[GeneratedRegex("[\r\n] +")]
private static partial Regex MyRegex2();
}
/// <summary>
/// A console capable of writing ANSI escape sequences.
/// </summary>
public static class CustomAnsiConsole
{
public static IAnsiConsole Console { get; set; } = AnsiConsole.Console;
public static void InitConsole(bool forceAnsi, bool noAnsiColor)
{
if (forceAnsi)
{
var ansiConsoleSettings = new AnsiConsoleSettings();
if (noAnsiColor)
{
ansiConsoleSettings.Out = new AnsiConsoleOutput(new NonAnsiWriter());
}
ansiConsoleSettings.Interactive = InteractionSupport.Yes;
ansiConsoleSettings.Ansi = AnsiSupport.Yes;
Console = AnsiConsole.Create(ansiConsoleSettings);
Console.Profile.Width = int.MaxValue;
}
else
{
var ansiConsoleSettings = new AnsiConsoleSettings();
if (noAnsiColor)
{
ansiConsoleSettings.Out = new AnsiConsoleOutput(new NonAnsiWriter());
}
Console = AnsiConsole.Create(ansiConsoleSettings);
}
}
/// <summary>
/// Writes the specified markup to the console.
/// </summary>
/// <param name="value">The value to write.</param>
public static void Markup(string value)
{
Console.Markup(value);
}
/// <summary>
/// Writes the specified markup, followed by the current line terminator, to the console.
/// </summary>
/// <param name="value">The value to write.</param>
public static void MarkupLine(string value)
{
Console.MarkupLine(value);
}
}

10
src/N_m3u8DL-RE.Common/Log/LogLevel.cs Normal file
View File

@ -0,0 +1,10 @@
namespace N_m3u8DL_RE.Common.Log;
public enum LogLevel
{
OFF,
ERROR,
WARN,
INFO,
DEBUG,
}

226
src/N_m3u8DL-RE.Common/Log/Logger.cs Normal file
View File

@ -0,0 +1,226 @@
using Spectre.Console;
using System.Text;
using System.Text.RegularExpressions;
namespace N_m3u8DL_RE.Common.Log;
public static partial class Logger
{
[GeneratedRegex("{}")]
private static partial Regex VarsRepRegex();
/// <summary>
/// 日志级别，默认为INFO
/// </summary>
public static LogLevel LogLevel { get; set; } = LogLevel.INFO;
/// <summary>
/// 是否写出日志文件
/// </summary>
public static bool IsWriteFile { get; set; } = true;
/// <summary>
/// 本次运行日志文件所在位置
/// </summary>
private static string? LogFilePath { get; set; }
// 读写锁
static ReaderWriterLockSlim LogWriteLock = new ReaderWriterLockSlim();
public static void InitLogFile()
{
if (!IsWriteFile) return;
try
{
var logDir = Path.GetDirectoryName(Environment.ProcessPath) + "/Logs";
if (!Directory.Exists(logDir))
{
Directory.CreateDirectory(logDir);
}
var now = DateTime.Now;
LogFilePath = Path.Combine(logDir, now.ToString("yyyy-MM-dd_HH-mm-ss-fff") + ".log");
int index = 1;
var fileName = Path.GetFileNameWithoutExtension(LogFilePath);
string init = "LOG " + now.ToString("yyyy/MM/dd") + Environment.NewLine
+ "Save Path: " + Path.GetDirectoryName(LogFilePath) + Environment.NewLine
+ "Task Start: " + now.ToString("yyyy/MM/dd HH:mm:ss") + Environment.NewLine
+ "Task CommandLine: " + Environment.CommandLine;
init += $"{Environment.NewLine}{Environment.NewLine}";
// 若文件存在则加序号
while (File.Exists(LogFilePath))
{
LogFilePath = Path.Combine(Path.GetDirectoryName(LogFilePath)!, $"{fileName}-{index++}.log");
}
File.WriteAllText(LogFilePath, init, Encoding.UTF8);
}
catch (Exception ex)
{
Error($"Init log failed! {ex.Message.RemoveMarkup()}");
}
}
private static string GetCurrTime()
{
return DateTime.Now.ToString("HH:mm:ss.fff");
}
private static void HandleLog(string write, string subWrite = "")
{
try
{
if (subWrite == "")
{
CustomAnsiConsole.MarkupLine(write);
}
else
{
CustomAnsiConsole.Markup(write);
Console.WriteLine(subWrite);
}
if (!IsWriteFile || !File.Exists(LogFilePath)) return;
var plain = write.RemoveMarkup() + subWrite.RemoveMarkup();
try
{
// 进入写入
LogWriteLock.EnterWriteLock();
using (StreamWriter sw = File.AppendText(LogFilePath))
{
sw.WriteLine(plain);
}
}
finally
{
// 释放占用
LogWriteLock.ExitWriteLock();
}
}
catch (Exception)
{
Console.WriteLine("Failed to write: " + write);
}
}
private static string ReplaceVars(string data, params object[] ps)
{
for (int i = 0; i < ps.Length; i++)
{
data = VarsRepRegex().Replace(data, $"{ps[i]}", 1);
}
return data;
}
public static void Info(string data, params object[] ps)
{
if (LogLevel < LogLevel.INFO) return;
data = ReplaceVars(data, ps);
var write = GetCurrTime() + " " + "[underline #548c26]INFO[/] : ";
HandleLog(write, data);
}
public static void InfoMarkUp(string data, params object[] ps)
{
if (LogLevel < LogLevel.INFO) return;
data = ReplaceVars(data, ps);
var write = GetCurrTime() + " " + "[underline #548c26]INFO[/] : " + data;
HandleLog(write);
}
public static void Debug(string data, params object[] ps)
{
if (LogLevel < LogLevel.DEBUG) return;
data = ReplaceVars(data, ps);
var write = GetCurrTime() + " " + "[underline grey]DEBUG[/]: ";
HandleLog(write, data);
}
public static void DebugMarkUp(string data, params object[] ps)
{
if (LogLevel < LogLevel.DEBUG) return;
data = ReplaceVars(data, ps);
var write = GetCurrTime() + " " + "[underline grey]DEBUG[/]: " + data;
HandleLog(write);
}
public static void Warn(string data, params object[] ps)
{
if (LogLevel < LogLevel.WARN) return;
data = ReplaceVars(data, ps);
var write = GetCurrTime() + " " + "[underline #a89022]WARN[/] : ";
HandleLog(write, data);
}
public static void WarnMarkUp(string data, params object[] ps)
{
if (LogLevel < LogLevel.WARN) return;
data = ReplaceVars(data, ps);
var write = GetCurrTime() + " " + "[underline #a89022]WARN[/] : " + data;
HandleLog(write);
}
public static void Error(string data, params object[] ps)
{
if (LogLevel < LogLevel.ERROR) return;
data = ReplaceVars(data, ps);
var write = GetCurrTime() + " " + "[underline red1]ERROR[/]: ";
HandleLog(write, data);
}
public static void ErrorMarkUp(string data, params object[] ps)
{
if (LogLevel < LogLevel.ERROR) return;
data = ReplaceVars(data, ps);
var write = GetCurrTime() + " " + "[underline red1]ERROR[/]: " + data;
HandleLog(write);
}
public static void ErrorMarkUp(Exception exception)
{
string data = exception.Message.EscapeMarkup();
if (LogLevel >= LogLevel.ERROR)
{
data = exception.ToString().EscapeMarkup();
}
ErrorMarkUp(data);
}
/// <summary>
/// This thing will only write to the log file.
/// </summary>
/// <param name="data"></param>
/// <param name="ps"></param>
public static void Extra(string data, params object[] ps)
{
if (!IsWriteFile || !File.Exists(LogFilePath)) return;
data = ReplaceVars(data, ps);
var plain = GetCurrTime() + " " + "EXTRA: " + data.RemoveMarkup();
try
{
// 进入写入
LogWriteLock.EnterWriteLock();
using (StreamWriter sw = File.AppendText(LogFilePath))
{
sw.WriteLine(plain, Encoding.UTF8);
}
}
finally
{
// 释放占用
LogWriteLock.ExitWriteLock();
}
}
}

16
src/N_m3u8DL-RE.Common/N_m3u8DL-RE.Common.csproj Normal file
View File

@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>library</OutputType>
<TargetFramework>net9.0</TargetFramework>
<RootNamespace>N_m3u8DL_RE.Common</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Spectre.Console" Version="0.49.2-preview.0.50" />
</ItemGroup>
</Project>

150
src/N_m3u8DL-RE.Common/Resource/ResString.cs Normal file
View File

@ -0,0 +1,150 @@
namespace N_m3u8DL_RE.Common.Resource;
public static class ResString
{
public static string CurrentLoc { get; set; } = "en-US";
public static readonly string ReLiveTs = "<RE_LIVE_TS>";
public static string singleFileRealtimeDecryptWarn => GetText("singleFileRealtimeDecryptWarn");
public static string singleFileSplitWarn => GetText("singleFileSplitWarn");
public static string customRangeWarn => GetText("customRangeWarn");
public static string customRangeFound => GetText("customRangeFound");
public static string customAdKeywordsFound => GetText("customAdKeywordsFound");
public static string customRangeInvalid => GetText("customRangeInvalid");
public static string consoleRedirected => GetText("consoleRedirected");
public static string autoBinaryMerge => GetText("autoBinaryMerge");
public static string autoBinaryMerge2 => GetText("autoBinaryMerge2");
public static string autoBinaryMerge3 => GetText("autoBinaryMerge3");
public static string autoBinaryMerge4 => GetText("autoBinaryMerge4");
public static string autoBinaryMerge5 => GetText("autoBinaryMerge5");
public static string autoBinaryMerge6 => GetText("autoBinaryMerge6");
public static string badM3u8 => GetText("badM3u8");
public static string binaryMerge => GetText("binaryMerge");
public static string checkingLast => GetText("checkingLast");
public static string cmd_appendUrlParams => GetText("cmd_appendUrlParams");
public static string cmd_autoSelect => GetText("cmd_autoSelect");
public static string cmd_disableUpdateCheck => GetText("cmd_disableUpdateCheck");
public static string cmd_binaryMerge => GetText("cmd_binaryMerge");
public static string cmd_useFFmpegConcatDemuxer => GetText("cmd_useFFmpegConcatDemuxer");
public static string cmd_checkSegmentsCount => GetText("cmd_checkSegmentsCount");
public static string cmd_decryptionBinaryPath => GetText("cmd_decryptionBinaryPath");
public static string cmd_delAfterDone => GetText("cmd_delAfterDone");
public static string cmd_ffmpegBinaryPath => GetText("cmd_ffmpegBinaryPath");
public static string cmd_mkvmergeBinaryPath => GetText("cmd_mkvmergeBinaryPath");
public static string cmd_baseUrl => GetText("cmd_baseUrl");
public static string cmd_maxSpeed => GetText("cmd_maxSpeed");
public static string cmd_adKeyword => GetText("cmd_adKeyword");
public static string cmd_moreHelp => GetText("cmd_moreHelp");
public static string cmd_header => GetText("cmd_header");
public static string cmd_muxImport => GetText("cmd_muxImport");
public static string cmd_muxImport_more => GetText("cmd_muxImport_more");
public static string cmd_selectVideo => GetText("cmd_selectVideo");
public static string cmd_dropVideo => GetText("cmd_dropVideo");
public static string cmd_selectVideo_more => GetText("cmd_selectVideo_more");
public static string cmd_selectAudio => GetText("cmd_selectAudio");
public static string cmd_dropAudio => GetText("cmd_dropAudio");
public static string cmd_selectAudio_more => GetText("cmd_selectAudio_more");
public static string cmd_selectSubtitle => GetText("cmd_selectSubtitle");
public static string cmd_dropSubtitle => GetText("cmd_dropSubtitle");
public static string cmd_selectSubtitle_more => GetText("cmd_selectSubtitle_more");
public static string cmd_custom_range => GetText("cmd_custom_range");
public static string cmd_customHLSMethod => GetText("cmd_customHLSMethod");
public static string cmd_customHLSKey => GetText("cmd_customHLSKey");
public static string cmd_customHLSIv => GetText("cmd_customHLSIv");
public static string cmd_Input => GetText("cmd_Input");
public static string cmd_forceAnsiConsole => GetText("cmd_forceAnsiConsole");
public static string cmd_noAnsiColor => GetText("cmd_noAnsiColor");
public static string cmd_keys => GetText("cmd_keys");
public static string cmd_keyText => GetText("cmd_keyText");
public static string cmd_loadKeyFailed => GetText("cmd_loadKeyFailed");
public static string cmd_logLevel => GetText("cmd_logLevel");
public static string cmd_MP4RealTimeDecryption => GetText("cmd_MP4RealTimeDecryption");
public static string cmd_saveDir => GetText("cmd_saveDir");
public static string cmd_saveName => GetText("cmd_saveName");
public static string cmd_savePattern => GetText("cmd_savePattern");
public static string cmd_skipDownload => GetText("cmd_skipDownload");
public static string cmd_noDateInfo => GetText("cmd_noDateInfo");
public static string cmd_noLog => GetText("cmd_noLog");
public static string cmd_allowHlsMultiExtMap => GetText("cmd_allowHlsMultiExtMap");
public static string cmd_skipMerge => GetText("cmd_skipMerge");
public static string cmd_subFormat => GetText("cmd_subFormat");
public static string cmd_subOnly => GetText("cmd_subOnly");
public static string cmd_subtitleFix => GetText("cmd_subtitleFix");
public static string cmd_threadCount => GetText("cmd_threadCount");
public static string cmd_downloadRetryCount => GetText("cmd_downloadRetryCount");
public static string cmd_httpRequestTimeout => GetText("cmd_httpRequestTimeout");
public static string cmd_tmpDir => GetText("cmd_tmpDir");
public static string cmd_uiLanguage => GetText("cmd_uiLanguage");
public static string cmd_urlProcessorArgs => GetText("cmd_urlProcessorArgs");
public static string cmd_useShakaPackager => GetText("cmd_useShakaPackager");
public static string cmd_decryptionEngine => GetText("cmd_decryptionEngine");
public static string cmd_concurrentDownload => GetText("cmd_concurrentDownload");
public static string cmd_useSystemProxy => GetText("cmd_useSystemProxy");
public static string cmd_customProxy => GetText("cmd_customProxy");
public static string cmd_customRange => GetText("cmd_customRange");
public static string cmd_liveKeepSegments => GetText("cmd_liveKeepSegments");
public static string cmd_livePipeMux => GetText("cmd_livePipeMux");
public static string cmd_liveRecordLimit => GetText("cmd_liveRecordLimit");
public static string cmd_taskStartAt => GetText("cmd_taskStartAt");
public static string cmd_liveWaitTime => GetText("cmd_liveWaitTime");
public static string cmd_liveTakeCount => GetText("cmd_liveTakeCount");
public static string cmd_liveFixVttByAudio => GetText("cmd_liveFixVttByAudio");
public static string cmd_liveRealTimeMerge => GetText("cmd_liveRealTimeMerge");
public static string cmd_livePerformAsVod => GetText("cmd_livePerformAsVod");
public static string cmd_muxAfterDone => GetText("cmd_muxAfterDone");
public static string cmd_muxAfterDone_more => GetText("cmd_muxAfterDone_more");
public static string cmd_writeMetaJson => GetText("cmd_writeMetaJson");
public static string liveLimit => GetText("liveLimit");
public static string realTimeDecMessage => GetText("realTimeDecMessage");
public static string liveLimitReached => GetText("liveLimitReached");
public static string saveName => GetText("saveName");
public static string taskStartAt => GetText("taskStartAt");
public static string namedPipeCreated => GetText("namedPipeCreated");
public static string namedPipeMux => GetText("namedPipeMux");
public static string partMerge => GetText("partMerge");
public static string fetch => GetText("fetch");
public static string ffmpegMerge => GetText("ffmpegMerge");
public static string ffmpegNotFound => GetText("ffmpegNotFound");
public static string mkvmergeNotFound => GetText("mkvmergeNotFound");
public static string mp4decryptNotFound => GetText("mp4decryptNotFound");
public static string shakaPackagerNotFound => GetText("shakaPackagerNotFound");
public static string fixingTTML => GetText("fixingTTML");
public static string fixingTTMLmp4 => GetText("fixingTTMLmp4");
public static string fixingVTT => GetText("fixingVTT");
public static string fixingVTTmp4 => GetText("fixingVTTmp4");
public static string keyProcessorNotFound => GetText("keyProcessorNotFound");
public static string liveFound => GetText("liveFound");
public static string loadingUrl => GetText("loadingUrl");
public static string masterM3u8Found => GetText("masterM3u8Found");
public static string allowHlsMultiExtMap => GetText("allowHlsMultiExtMap");
public static string matchDASH => GetText("matchDASH");
public static string matchMSS => GetText("matchMSS");
public static string matchTS => GetText("matchTS");
public static string matchHLS => GetText("matchHLS");
public static string notSupported => GetText("notSupported");
public static string parsingStream => GetText("parsingStream");
public static string promptChoiceText => GetText("promptChoiceText");
public static string promptInfo => GetText("promptInfo");
public static string promptTitle => GetText("promptTitle");
public static string readingInfo => GetText("readingInfo");
public static string searchKey => GetText("searchKey");
public static string decryptionFailed => GetText("decryptionFailed");
public static string segmentCountCheckNotPass => GetText("segmentCountCheckNotPass");
public static string selectedStream => GetText("selectedStream");
public static string startDownloading => GetText("startDownloading");
public static string streamsInfo => GetText("streamsInfo");
public static string writeJson => GetText("writeJson");
public static string noStreamsToDownload => GetText("noStreamsToDownload");
public static string newVersionFound => GetText("newVersionFound");
public static string processImageSub => GetText("processImageSub");
private static string GetText(string key)
{
if (!StaticText.LANG_DIC.TryGetValue(key, out var textObj))
return "<...LANG TEXT MISSING...>";
if (CurrentLoc is "zh-CN" or "zh-SG" or "zh-Hans")
return textObj.ZH_CN;
return CurrentLoc.StartsWith("zh-") ? textObj.ZH_TW : textObj.EN_US;
}
}

969
src/N_m3u8DL-RE.Common/Resource/StaticText.cs Normal file
View File

@ -0,0 +1,969 @@
namespace N_m3u8DL_RE.Common.Resource;
internal static class StaticText
{
public static readonly Dictionary<string, TextContainer> LANG_DIC = new()
{
["singleFileSplitWarn"] = new TextContainer
(
zhCN: "整段文件已被自动切割为小分片以加速下载",
zhTW: "整段文件已被自動切割為小分片以加速下載",
enUS: "The entire file has been cut into small segments to accelerate"
),
["singleFileRealtimeDecryptWarn"] = new TextContainer
(
zhCN: "实时解密已被强制关闭",
zhTW: "即時解密已被強制關閉",
enUS: "Real-time decryption has been disabled"
),
["cmd_forceAnsiConsole"] = new TextContainer
(
zhCN: "强制认定终端为支持ANSI且可交互的终端",
zhTW: "強制認定終端為支援ANSI且可交往的終端",
enUS: "Force assuming the terminal is ANSI-compatible and interactive"
),
["cmd_noAnsiColor"] = new TextContainer
(
zhCN: "去除ANSI颜色",
zhTW: "關閉ANSI顏色",
enUS: "Remove ANSI colors"
),
["customRangeWarn"] = new TextContainer
(
zhCN: "请注意，自定义下载范围有时会导致音画不同步",
zhTW: "請注意，自定義下載範圍有時會導致音畫不同步",
enUS: "Please note that custom range may sometimes result in audio and video being out of sync"
),
["customRangeInvalid"] = new TextContainer
(
zhCN: "自定义下载范围无效",
zhTW: "自定義下載範圍無效",
enUS: "User customed range invalid"
),
["customAdKeywordsFound"] = new TextContainer
(
zhCN: "用户自定义广告分片URL关键字：",
zhTW: "用戶自定義廣告分片URL關鍵字：",
enUS: "User customed Ad keyword: "
),
["customRangeFound"] = new TextContainer
(
zhCN: "用户自定义下载范围：",
zhTW: "用戶自定義下載範圍：",
enUS: "User customed range: "
),
["consoleRedirected"] = new TextContainer
(
zhCN: "输出被重定向, 将清除ANSI颜色",
zhTW: "輸出被重定向, 將清除ANSI顏色",
enUS: "Output is redirected, ANSI colors are cleared."
),
["processImageSub"] = new TextContainer
(
zhCN: "正在处理图形字幕",
zhTW: "正在處理圖形字幕",
enUS: "Processing Image Sub"
),
["newVersionFound"] = new TextContainer
(
zhCN: "检测到新版本，请尽快升级！",
zhTW: "檢測到新版本，請盡快升級！",
enUS: "New version detected!"
),
["namedPipeCreated"] = new TextContainer
(
zhCN: "已创建命名管道：",
zhTW: "已創建命名管道：",
enUS: "Named pipe created: "
),
["namedPipeMux"] = new TextContainer
(
zhCN: "通过命名管道混流到",
zhTW: "通過命名管道混流到",
enUS: "Mux with named pipe, to"
),
["taskStartAt"] = new TextContainer
(
zhCN: "程序将等待，直到：",
zhTW: "程序將等待，直到：",
enUS: "The program will wait until: "
),
["autoBinaryMerge"] = new TextContainer
(
zhCN: "检测到fMP4，自动开启二进制合并",
zhTW: "檢測到fMP4，自動開啟二進位制合併",
enUS: "fMP4 is detected, binary merging is automatically enabled"
),
["autoBinaryMerge2"] = new TextContainer
(
zhCN: "检测到杜比视界内容，自动开启二进制合并",
zhTW: "檢測到杜比視界內容，自動開啟二進位制合併",
enUS: "Dolby Vision content is detected, binary merging is automatically enabled"
),
["autoBinaryMerge3"] = new TextContainer
(
zhCN: "检测到无法识别的加密方式，自动开启二进制合并",
zhTW: "檢測到無法識別的加密方式，自動開啟二進位制合併",
enUS: "An unrecognized encryption method is detected, binary merging is automatically enabled"
),
["autoBinaryMerge4"] = new TextContainer
(
zhCN: "检测到CENC加密方式，自动开启二进制合并",
zhTW: "檢測到CENC加密方式，自動開啟二進位制合併",
enUS: "When CENC encryption is detected, binary merging is automatically enabled"
),
["autoBinaryMerge5"] = new TextContainer
(
zhCN: "检测到杜比视界内容，混流功能已禁用",
zhTW: "檢測到杜比視界內容，混流功能已禁用",
enUS: "Dolby Vision content is detected, mux after done is automatically disabled"
),
["autoBinaryMerge6"] = new TextContainer
(
zhCN: "你已开启下载完成后混流，自动开启二进制合并",
zhTW: "你已開啟下載完成後混流，自動開啟二進制合併",
enUS: "MuxAfterDone is detected, binary merging is automatically enabled"
),
["badM3u8"] = new TextContainer
(
zhCN: "错误的m3u8",
zhTW: "錯誤的m3u8",
enUS: "Bad m3u8"
),
["binaryMerge"] = new TextContainer
(
zhCN: "二进制合并中...",
zhTW: "二進位制合併中...",
enUS: "Binary merging..."
),
["checkingLast"] = new TextContainer
(
zhCN: "验证最后一个分片有效性",
zhTW: "驗證最後一個分片有效性",
enUS: "Verifying the validity of the last segment"
),
["cmd_baseUrl"] = new TextContainer
(
zhCN: "设置BaseURL",
zhTW: "設置BaseURL",
enUS: "Set BaseURL"
),
["cmd_maxSpeed"] = new TextContainer
(
zhCN: "设置限速，单位支持 Mbps 或 Kbps，如：15M 100K",
zhTW: "設置限速，單位支持 Mbps 或 Kbps，如：15M 100K",
enUS: "Set speed limit, Mbps or Kbps, for example: 15M 100K."
),
["cmd_noDateInfo"] = new TextContainer
(
zhCN: "混流时不写入日期信息",
zhTW: "混流時不寫入日期訊息",
enUS: "Date information is not written during muxing"
),
["cmd_noLog"] = new TextContainer
(
zhCN: "关闭日志文件输出",
zhTW: "關閉日誌文件輸出",
enUS: "Disable log file output"
),
["cmd_allowHlsMultiExtMap"] = new TextContainer
(
zhCN: "允许HLS中的多个#EXT-X-MAP(实验性)",
zhTW: "允許HLS中的多個#EXT-X-MAP(實驗性)",
enUS: "Allow multiple #EXT-X-MAP in HLS (experimental)"
),
["cmd_appendUrlParams"] = new TextContainer
(
zhCN: "将输入Url的Params添加至分片, 对某些网站很有用, 例如 kakao.com",
zhTW: "將輸入Url的Params添加至分片, 對某些網站很有用, 例如 kakao.com",
enUS: "Add Params of input Url to segments, useful for some websites, such as kakao.com"
),
["cmd_autoSelect"] = new TextContainer
(
zhCN: "自动选择所有类型的最佳轨道",
zhTW: "自動選擇所有類型的最佳軌道",
enUS: "Automatically selects the best tracks of all types"
),
["cmd_disableUpdateCheck"] = new TextContainer
(
zhCN: "禁用版本更新检测",
zhTW: "禁用版本更新檢測",
enUS: "Disable version update check"
),
["cmd_binaryMerge"] = new TextContainer
(
zhCN: "二进制合并",
zhTW: "二進位制合併",
enUS: "Binary merge"
),
["cmd_useFFmpegConcatDemuxer"] = new TextContainer
(
zhCN: "使用 ffmpeg 合并时，使用 concat 分离器而非 concat 协议",
zhTW: "使用 ffmpeg 合併時，使用 concat 分離器而非 concat 協議",
enUS: "When merging with ffmpeg, use the concat demuxer instead of the concat protocol"
),
["cmd_checkSegmentsCount"] = new TextContainer
(
zhCN: "检测实际下载的分片数量和预期数量是否匹配",
zhTW: "檢測實際下載的分片數量和預期數量是否匹配",
enUS: "Check if the actual number of segments downloaded matches the expected number"
),
["cmd_downloadRetryCount"] = new TextContainer
(
zhCN: "每个分片下载异常时的重试次数",
zhTW: "每個分片下載異常時的重試次數",
enUS: "The number of retries when download segment error"
),
["cmd_httpRequestTimeout"] = new TextContainer
(
zhCN: "HTTP请求的超时时间(秒)",
zhTW: "HTTP請求的超時時間(秒)",
enUS: "Timeout duration for HTTP requests (in seconds)"
),
["cmd_decryptionBinaryPath"] = new TextContainer
(
zhCN: @"MP4解密所用工具的全路径, 例如 C:\Tools\mp4decrypt.exe",
zhTW: @"MP4解密所用工具的全路徑, 例如 C:\Tools\mp4decrypt.exe",
enUS: @"Full path to the tool used for MP4 decryption, like C:\Tools\mp4decrypt.exe"
),
["cmd_delAfterDone"] = new TextContainer
(
zhCN: "完成后删除临时文件",
zhTW: "完成後刪除臨時文件",
enUS: "Delete temporary files when done"
),
["cmd_ffmpegBinaryPath"] = new TextContainer
(
zhCN: @"ffmpeg可执行程序全路径, 例如 C:\Tools\ffmpeg.exe",
zhTW: @"ffmpeg可執行程序全路徑, 例如 C:\Tools\ffmpeg.exe",
enUS: @"Full path to the ffmpeg binary, like C:\Tools\ffmpeg.exe"
),
["cmd_mkvmergeBinaryPath"] = new TextContainer
(
zhCN: @"mkvmerge可执行程序全路径, 例如 C:\Tools\mkvmerge.exe",
zhTW: @"mkvmerge可執行程序全路徑, 例如 C:\Tools\mkvmerge.exe",
enUS: @"Full path to the mkvmerge binary, like C:\Tools\mkvmerge.exe"
),
["cmd_liveFixVttByAudio"] = new TextContainer
(
zhCN: "通过读取音频文件的起始时间修正VTT字幕",
zhTW: "透過讀取音訊檔案的起始時間修正VTT字幕",
enUS: "Correct VTT sub by reading the start time of the audio file"
),
["cmd_header"] = new TextContainer
(
zhCN: "为HTTP请求设置特定的请求头, 例如:\r\n-H \"Cookie: mycookie\" -H \"User-Agent: iOS\"",
zhTW: "為HTTP請求設置特定的請求頭, 例如:\r\n-H \"Cookie: mycookie\" -H \"User-Agent: iOS\"",
enUS: "Pass custom header(s) to server, Example:\r\n-H \"Cookie: mycookie\" -H \"User-Agent: iOS\""
),
["cmd_Input"] = new TextContainer
(
zhCN: "链接或文件",
zhTW: "連結或文件",
enUS: "Input Url or File"
),
["cmd_keys"] = new TextContainer
(
zhCN: "设置解密密钥, 程序调用mp4decrpyt/shaka-packager/ffmpeg进行解密. 格式:\r\n--key KID1:KEY1 --key KID2:KEY2\r\n对于KEY相同的情况可以直接输入 --key KEY",
zhTW: "設置解密密鑰, 程序調用mp4decrpyt/shaka-packager/ffmpeg進行解密. 格式:\r\n--key KID1:KEY1 --key KID2:KEY2\r\n對於KEY相同的情況可以直接輸入 --key KEY",
enUS: "Set decryption key(s) to mp4decrypt/shaka-packager/ffmpeg. format:\r\n--key KID1:KEY1 --key KID2:KEY2\r\nor use --key KEY if all tracks share the same key."
),
["cmd_keyText"] = new TextContainer
(
zhCN: "设置密钥文件,程序将从文件中按KID搜寻KEY以解密.(不建议使用特大文件)",
zhTW: "設置密鑰文件,程序將從文件中按KID搜尋KEY以解密.(不建議使用特大文件)",
enUS: "Set the kid-key file, the program will search the KEY with KID from the file.(Very large file are not recommended)"
),
["cmd_loadKeyFailed"] = new TextContainer
(
zhCN: "获取KEY失败，忽略读取.",
zhTW: "獲取KEY失敗，忽略讀取.",
enUS: "Failed to get KEY, ignore."
),
["cmd_logLevel"] = new TextContainer
(
zhCN: "设置日志级别",
zhTW: "設置日誌級別",
enUS: "Set log level"
),
["cmd_MP4RealTimeDecryption"] = new TextContainer
(
zhCN: "实时解密MP4分片",
zhTW: "即時解密MP4分片",
enUS: "Decrypt MP4 segments in real time"
),
["cmd_saveDir"] = new TextContainer
(
zhCN: "设置输出目录",
zhTW: "設置輸出目錄",
enUS: "Set output directory"
),
["cmd_saveName"] = new TextContainer
(
zhCN: "设置保存文件名",
zhTW: "設置保存檔案名",
enUS: "Set output filename"
),
["cmd_savePattern"] = new TextContainer
(
zhCN: "设置保存文件命名模板, 支持使用变量",
zhTW: "",
enUS: ""
),
["cmd_skipDownload"] = new TextContainer
(
zhCN: "跳过下载",
zhTW: "跳過下載",
enUS: "Skip download"
),
["cmd_skipMerge"] = new TextContainer
(
zhCN: "跳过合并分片",
zhTW: "跳過合併分片",
enUS: "Skip segments merge"
),
["cmd_subFormat"] = new TextContainer
(
zhCN: "字幕输出类型",
zhTW: "字幕輸出類型",
enUS: "Subtitle output format"
),
["cmd_subOnly"] = new TextContainer
(
zhCN: "只选取字幕轨道",
zhTW: "只選取字幕軌道",
enUS: "Select only subtitle tracks"
),
["cmd_subtitleFix"] = new TextContainer
(
zhCN: "自动修正字幕",
zhTW: "自動修正字幕",
enUS: "Automatically fix subtitles"
),
["cmd_threadCount"] = new TextContainer
(
zhCN: "设置下载线程数",
zhTW: "設置下載執行緒數",
enUS: "Set download thread count"
),
["cmd_tmpDir"] = new TextContainer
(
zhCN: "设置临时文件存储目录",
zhTW: "設置臨時文件儲存目錄",
enUS: "Set temporary file directory"
),
["cmd_uiLanguage"] = new TextContainer
(
zhCN: "设置UI语言",
zhTW: "設置UI語言",
enUS: "Set UI language"
),
["cmd_moreHelp"] = new TextContainer
(
zhCN: "查看某个选项的详细帮助信息",
zhTW: "查看某個選項的詳細幫助訊息",
enUS: "Set more help info about one option"
),
["cmd_urlProcessorArgs"] = new TextContainer
(
zhCN: "此字符串将直接传递给URL Processor",
zhTW: "此字符串將直接傳遞給URL Processor",
enUS: "Give these arguments to the URL Processors."
),
["cmd_liveRealTimeMerge"] = new TextContainer
(
zhCN: "录制直播时实时合并",
zhTW: "錄製直播時即時合併",
enUS: "Real-time merge into file when recording live"
),
["cmd_customProxy"] = new TextContainer
(
zhCN: "设置请求代理, 如 http://127.0.0.1:8888",
zhTW: "設置請求代理, 如 http://127.0.0.1:8888",
enUS: "Set web request proxy, like http://127.0.0.1:8888"
),
["cmd_customRange"] = new TextContainer
(
zhCN: "仅下载部分分片. 输入 \"--morehelp custom-range\" 以查看详细信息",
zhTW: "僅下載部分分片. 輸入 \"--morehelp custom-range\" 以查看詳細訊息",
enUS: "Download only part of the segments. Use \"--morehelp custom-range\" for more details"
),
["cmd_useSystemProxy"] = new TextContainer
(
zhCN: "使用系统默认代理",
zhTW: "使用系統默認代理",
enUS: "Use system default proxy"
),
["cmd_livePerformAsVod"] = new TextContainer
(
zhCN: "以点播方式下载直播流",
zhTW: "以點播方式下載直播流",
enUS: "Download live streams as vod"
),
["cmd_liveWaitTime"] = new TextContainer
(
zhCN: "手动设置直播列表刷新间隔",
zhTW: "手動設置直播列表刷新間隔",
enUS: "Manually set the live playlist refresh interval"
),
["cmd_adKeyword"] = new TextContainer
(
zhCN: "设置广告分片的URL关键字(正则表达式)",
zhTW: "設置廣告分片的URL關鍵字(正則表達式)",
enUS: "Set URL keywords (regular expressions) for AD segments"
),
["cmd_liveTakeCount"] = new TextContainer
(
zhCN: "手动设置录制直播时首次获取分片的数量",
zhTW: "手動設置錄製直播時首次獲取分片的數量",
enUS: "Manually set the number of segments downloaded for the first time when recording live"
),
["cmd_customHLSMethod"] = new TextContainer
(
zhCN: "指定HLS加密方式 (AES_128|AES_128_ECB|CENC|CHACHA20|NONE|SAMPLE_AES|SAMPLE_AES_CTR|UNKNOWN)",
zhTW: "指定HLS加密方式 (AES_128|AES_128_ECB|CENC|CHACHA20|NONE|SAMPLE_AES|SAMPLE_AES_CTR|UNKNOWN)",
enUS: "Set HLS encryption method (AES_128|AES_128_ECB|CENC|CHACHA20|NONE|SAMPLE_AES|SAMPLE_AES_CTR|UNKNOWN)"
),
["cmd_customHLSKey"] = new TextContainer
(
zhCN: "指定HLS解密KEY. 可以是文件, HEX或Base64",
zhTW: "指定HLS解密KEY. 可以是文件, HEX或Base64",
enUS: "Set the HLS decryption key. Can be file, HEX or Base64"
),
["cmd_customHLSIv"] = new TextContainer
(
zhCN: "指定HLS解密IV. 可以是文件, HEX或Base64",
zhTW: "指定HLS解密IV. 可以是文件, HEX或Base64",
enUS: "Set the HLS decryption iv. Can be file, HEX or Base64"
),
["cmd_livePipeMux"] = new TextContainer
(
zhCN: "录制直播并开启实时合并时通过管道+ffmpeg实时混流到TS文件",
zhTW: "錄製直播並開啟即時合併時通過管道+ffmpeg即時混流到TS文件",
enUS: "Real-time muxing to TS file through pipeline + ffmpeg (liveRealTimeMerge enabled)"
),
["cmd_liveKeepSegments"] = new TextContainer
(
zhCN: "录制直播并开启实时合并时依然保留分片",
zhTW: "錄製直播並開啟即時合併時依然保留分片",
enUS: "Keep segments when recording a live (liveRealTimeMerge enabled)"
),
["cmd_liveRecordLimit"] = new TextContainer
(
zhCN: "录制直播时的录制时长限制",
zhTW: "錄製直播時的錄製時長限制",
enUS: "Recording time limit when recording live"
),
["cmd_taskStartAt"] = new TextContainer
(
zhCN: "在此时间之前不会开始执行任务",
zhTW: "在此時間之前不會開始執行任務",
enUS: "Task execution will not start before this time"
),
["cmd_useShakaPackager"] = new TextContainer
(
zhCN: "解密时使用shaka-packager替代mp4decrypt",
zhTW: "解密時使用shaka-packager替代mp4decrypt",
enUS: "Use shaka-packager instead of mp4decrypt to decrypt"
),
["cmd_decryptionEngine"] = new TextContainer
(
zhCN: "设置解密时使用的第三方程序",
zhTW: "設置解密時使用的第三方程序",
enUS: "Set the third-party program used for decryption"
),
["cmd_concurrentDownload"] = new TextContainer
(
zhCN: "并发下载已选择的音频、视频和字幕",
zhTW: "並發下載已選擇的音訊、影片和字幕",
enUS: "Concurrently download the selected audio, video and subtitles"
),
["cmd_selectVideo"] = new TextContainer
(
zhCN: "通过正则表达式选择符合要求的视频流. 输入 \"--morehelp select-video\" 以查看详细信息",
zhTW: "通過正則表達式選擇符合要求的影片軌. 輸入 \"--morehelp select-video\" 以查看詳細訊息",
enUS: "Select video streams by regular expressions. Use \"--morehelp select-video\" for more details"
),
["cmd_dropVideo"] = new TextContainer
(
zhCN: "通过正则表达式去除符合要求的视频流.",
zhTW: "通過正則表達式去除符合要求的影片串流.",
enUS: "Drop video streams by regular expressions."
),
["cmd_selectVideo_more"] = new TextContainer
(
zhCN: "通过正则表达式选择符合要求的视频流. 你能够以:分隔形式指定如下参数:\r\n\r\n" +
"id=REGEX:lang=REGEX:name=REGEX:codecs=REGEX:res=REGEX:frame=REGEX\r\n" +
"segsMin=number:segsMax=number:ch=REGEX:range=REGEX:url=REGEX\r\n" +
"plistDurMin=hms:plistDurMax=hms:bwMin=int:bwMax=int:role=string:for=FOR\r\n\r\n" +
"* for=FOR: 选择方式. best[number], worst[number], all (默认: best)\r\n\r\n" +
"例如: \r\n" +
"# 选择最佳视频\r\n" +
"-sv best\r\n" +
"# 选择4K+HEVC视频\r\n" +
"-sv res=\"3840*\":codecs=hvc1:for=best\r\n" +
"# 选择长度大于1小时20分钟30秒的视频\r\n" +
"-sv plistDurMin=\"1h20m30s\":for=best\r\n" +
"-sv role=\"main\":for=best\r\n" +
"# 选择码率在800Kbps至1Mbps之间的视频\r\n" +
"-sv bwMin=800:bwMax=1000\r\n",
zhTW: "通過正則表達式選擇符合要求的影片軌. 你能夠以:分隔形式指定如下參數:\r\n\r\n" +
"id=REGEX:lang=REGEX:name=REGEX:codecs=REGEX:res=REGEX:frame=REGEX\r\n" +
"segsMin=number:segsMax=number:ch=REGEX:range=REGEX:url=REGEX\r\n" +
"plistDurMin=hms:plistDurMax=hms:bwMin=int:bwMax=int:role=string:for=FOR\r\n\r\n" +
"* for=FOR: 選擇方式. best[number], worst[number], all (默認: best)\r\n\r\n" +
"例如: \r\n" +
"# 選擇最佳影片\r\n" +
"-sv best\r\n" +
"# 選擇4K+HEVC影片\r\n" +
"-sv res=\"3840*\":codecs=hvc1:for=best\r\n" +
"# 選擇長度大於1小時20分鐘30秒的影片\r\n" +
"-sv plistDurMin=\"1h20m30s\":for=best\r\n" +
"-sv role=\"main\":for=best\r\n" +
"# 選擇碼率在800Kbps至1Mbps之間的影片\r\n" +
"-sv bwMin=800:bwMax=1000\r\n",
enUS: "Select video streams by regular expressions. OPTIONS is a colon separated list of:\r\n\r\n" +
"id=REGEX:lang=REGEX:name=REGEX:codecs=REGEX:res=REGEX:frame=REGEX\r\n" +
"segsMin=number:segsMax=number:ch=REGEX:range=REGEX:url=REGEX\r\n" +
"plistDurMin=hms:plistDurMax=hms:bwMin=int:bwMax=int:role=string:for=FOR\r\n\r\n" +
"* for=FOR: Select type. best[number], worst[number], all (Default: best)\r\n\r\n" +
"Examples: \r\n" +
"# select best video\r\n" +
"-sv best\r\n" +
"# select 4K+HEVC video\r\n" +
"-sv res=\"3840*\":codecs=hvc1:for=best\r\n" +
"# Select best video with duration longer than 1 hour 20 minutes 30 seconds\r\n" +
"-sv plistDurMin=\"1h20m30s\":for=best\r\n" +
"-sv role=\"main\":for=best\r\n" +
"# Select video with bandwidth between 800Kbps and 1Mbps\r\n" +
"-sv bwMin=800:bwMax=1000\r\n"
),
["cmd_selectAudio"] = new TextContainer
(
zhCN: "通过正则表达式选择符合要求的音频流. 输入 \"--morehelp select-audio\" 以查看详细信息",
zhTW: "通過正則表達式選擇符合要求的音軌. 輸入 \"--morehelp select-audio\" 以查看詳細訊息",
enUS: "Select audio streams by regular expressions. Use \"--morehelp select-audio\" for more details"
),
["cmd_dropAudio"] = new TextContainer
(
zhCN: "通过正则表达式去除符合要求的音频流.",
zhTW: "通過正則表達式去除符合要求的音軌.",
enUS: "Drop audio streams by regular expressions."
),
["cmd_selectAudio_more"] = new TextContainer
(
zhCN: "通过正则表达式选择符合要求的音频流. 参考 --select-video\r\n\r\n" +
"例如: \r\n" +
"# 选择所有音频\r\n" +
"-sa all\r\n" +
"# 选择最佳英语音轨\r\n" +
"-sa lang=en:for=best\r\n" +
"# 选择最佳的2条英语(或日语)音轨\r\n" +
"-sa lang=\"ja|en\":for=best2\r\n" +
"-sa role=\"main\":for=best\r\n",
zhTW: "通過正則表達式選擇符合要求的音軌. 參考 --select-video\r\n\r\n" +
"例如: \r\n" +
"# 選擇所有音訊\r\n" +
"-sa all\r\n" +
"# 選擇最佳英語音軌\r\n" +
"-sa lang=en:for=best\r\n" +
"# 選擇最佳的2條英語(或日語)音軌\r\n" +
"-sa lang=\"ja|en\":for=best2\r\n" +
"-sa role=\"main\":for=best\r\n",
enUS: "Select audio streams by regular expressions. ref --select-video\r\n\r\n" +
"Examples: \r\n" +
"# select all\r\n" +
"-sa all\r\n" +
"# select best eng audio\r\n" +
"-sa lang=en:for=best\r\n" +
"# select best 2, and language is ja or en\r\n" +
"-sa lang=\"ja|en\":for=best2\r\n" +
"-sa role=\"main\":for=best\r\n"
),
["cmd_selectSubtitle"] = new TextContainer
(
zhCN: "通过正则表达式选择符合要求的字幕流. 输入 \"--morehelp select-subtitle\" 以查看详细信息",
zhTW: "通過正則表達式選擇符合要求的字幕流. 輸入 \"--morehelp select-subtitle\" 以查看詳細訊息",
enUS: "Select subtitle streams by regular expressions. Use \"--morehelp select-subtitle\" for more details"
),
["cmd_dropSubtitle"] = new TextContainer
(
zhCN: "通过正则表达式去除符合要求的字幕流.",
zhTW: "通過正則表達式去除符合要求的字幕流.",
enUS: "Drop subtitle streams by regular expressions."
),
["cmd_custom_range"] = new TextContainer
(
zhCN: "下载点播内容时, 仅下载部分分片.\r\n\r\n" +
"例如: \r\n" +
"# 下载[0,10]共11个分片\r\n" +
"--custom-range 0-10\r\n" +
"# 下载从序号10开始的后续分片\r\n" +
"--custom-range 10-\r\n" +
"# 下载前100个分片\r\n" +
"--custom-range -99\r\n" +
"# 下载第5分钟到20分钟的内容\r\n" +
"--custom-range 05:00-20:00\r\n",
zhTW: "下載點播內容時, 僅下載部分分片.\r\n\r\n" +
"例如: \r\n" +
"# 下載[0,10]共11個分片\r\n" +
"--custom-range 0-10\r\n" +
"# 下載從序號10開始的後續分片\r\n" +
"--custom-range 10-\r\n" +
"# 下載前100個分片\r\n" +
"--custom-range -99\r\n" +
"# 下載第5分鐘到20分鐘的內容\r\n" +
"--custom-range 05:00-20:00\r\n",
enUS: "Download only part of the segments when downloading vod content.\r\n\r\n" +
"Examples: \r\n" +
"# Download [0,10], a total of 11 segments\r\n" +
"--custom-range 0-10\r\n" +
"# Download subsequent segments starting from index 10\r\n" +
"--custom-range 10-\r\n" +
"# Download the first 100 segments\r\n" +
"--custom-range -99\r\n" +
"# Download content from the 05:00 to 20:00\r\n" +
"--custom-range 05:00-20:00\r\n"
),
["cmd_selectSubtitle_more"] = new TextContainer
(
zhCN: "通过正则表达式选择符合要求的字幕流. 参考 --select-video\r\n\r\n" +
"例如: \r\n" +
"# 选择所有字幕\r\n" +
"-ss all\r\n" +
"# 选择所有带有\"中文\"的字幕\r\n" +
"-ss name=\"中文\":for=all\r\n",
zhTW: "通過正則表達式選擇符合要求的字幕流. 參考 --select-video\r\n\r\n" +
"例如: \r\n" +
"# 選擇所有字幕\r\n" +
"-ss all\r\n" +
"# 選擇所有帶有\"中文\"的字幕\r\n" +
"-ss name=\"中文\":for=all\r\n",
enUS: "Select subtitle streams by regular expressions. ref --select-video\r\n\r\n" +
"Examples: \r\n" +
"# select all subs\r\n" +
"-ss all\r\n" +
"# select all subs containing \"English\"\r\n" +
"-ss name=\"English\":for=all\r\n"
),
["cmd_muxAfterDone_more"] = new TextContainer
(
zhCN: "所有工作完成时尝试混流分离的音视频. 你能够以:分隔形式指定如下参数:\r\n\r\n" +
"* format=FORMAT: 指定混流容器 mkv, mp4, ts\r\n" +
"* muxer=MUXER: 指定混流程序 ffmpeg, mkvmerge (默认: ffmpeg)\r\n" +
"* bin_path=PATH: 指定程序路径 (默认: 自动寻找)\r\n" +
"* skip_sub=BOOL: 是否忽略字幕文件 (默认: false)\r\n" +
"* keep=BOOL: 混流完成是否保留文件 true, false (默认: false)\r\n\r\n" +
"例如: \r\n" +
"# 混流为mp4容器\r\n" +
"-M format=mp4\r\n" +
"# 使用mkvmerge, 自动寻找程序\r\n" +
"-M format=mkv:muxer=mkvmerge\r\n" +
"# 使用mkvmerge, 自定义程序路径\r\n" +
"-M format=mkv:muxer=mkvmerge:bin_path=\"C\\:\\Program Files\\MKVToolNix\\mkvmerge.exe\"\r\n",
zhTW: "所有工作完成時嘗試混流分離的影音. 你能夠以:分隔形式指定如下參數:\r\n\r\n" +
"* format=FORMAT: 指定混流容器 mkv, mp4, ts\r\n" +
"* muxer=MUXER: 指定混流程序 ffmpeg, mkvmerge (默認: ffmpeg)\r\n" +
"* bin_path=PATH: 指定程序路徑 (默認: 自動尋找)\r\n" +
"* skip_sub=BOOL: 是否忽略字幕文件 (默認: false)\r\n" +
"* keep=BOOL: 混流完成是否保留文件 true, false (默認: false)\r\n\r\n" +
"例如: \r\n" +
"# 混流為mp4容器\r\n" +
"-M format=mp4\r\n" +
"# 使用mkvmerge, 自動尋找程序\r\n" +
"-M format=mkv:muxer=mkvmerge\r\n" +
"# 使用mkvmerge, 自訂程序路徑\r\n" +
"-M format=mkv:muxer=mkvmerge:bin_path=\"C\\:\\Program Files\\MKVToolNix\\mkvmerge.exe\"\r\n",
enUS: "When all works is done, try to mux the downloaded streams. OPTIONS is a colon separated list of:\r\n\r\n" +
"* format=FORMAT: set container. mkv, mp4, ts\r\n" +
"* muxer=MUXER: set muxer. ffmpeg, mkvmerge (Default: ffmpeg)\r\n" +
"* bin_path=PATH: set binary file path. (Default: auto)\r\n" +
"* skip_sub=BOOL: set whether or not skip subtitle files (Default: false)\r\n" +
"* keep=BOOL: set whether or not keep files. true, false (Default: false)\r\n\r\n" +
"Examples: \r\n" +
"# mux to mp4\r\n" +
"-M format=mp4\r\n" +
"# use mkvmerge, auto detect bin path\r\n" +
"-M format=mkv:muxer=mkvmerge\r\n" +
"# use mkvmerge, set bin path\r\n" +
"-M format=mkv:muxer=mkvmerge:bin_path=\"C\\:\\Program Files\\MKVToolNix\\mkvmerge.exe\"\r\n"
),
["cmd_muxAfterDone"] = new TextContainer
(
zhCN: "所有工作完成时尝试混流分离的音视频. 输入 \"--morehelp mux-after-done\" 以查看详细信息",
zhTW: "所有工作完成時嘗試混流分離的影音. 輸入 \"--morehelp mux-after-done\" 以查看詳細訊息",
enUS: "When all works is done, try to mux the downloaded streams. Use \"--morehelp mux-after-done\" for more details"
),
["cmd_muxImport"] = new TextContainer
(
zhCN: "混流时引入外部媒体文件. 输入 \"--morehelp mux-import\" 以查看详细信息",
zhTW: "混流時引入外部媒體檔案. 輸入 \"--morehelp mux-import\" 以查看詳細訊息",
enUS: "When MuxAfterDone enabled, allow to import local media files. Use \"--morehelp mux-import\" for more details"
),
["cmd_muxImport_more"] = new TextContainer
(
zhCN: "混流时引入外部媒体文件. 你能够以:分隔形式指定如下参数:\r\n\r\n" +
"* path=PATH: 指定媒体文件路径\r\n" +
"* lang=CODE: 指定媒体文件语言代码 (非必须)\r\n" +
"* name=NAME: 指定媒体文件描述信息 (非必须)\r\n\r\n" +
"例如: \r\n" +
"# 引入外部字幕\r\n" +
"--mux-import path=zh-Hans.srt:lang=chi:name=\"中文 (简体)\"\r\n" +
"# 引入外部音轨+字幕\r\n" +
"--mux-import path=\"D\\:\\media\\atmos.m4a\":lang=eng:name=\"English Description Audio\" --mux-import path=\"D\\:\\media\\eng.vtt\":lang=eng:name=\"English (Description)\"",
zhTW: "混流時引入外部媒體檔案. 你能夠以:分隔形式指定如下參數:\r\n\r\n" +
"* path=PATH: 指定媒體檔案路徑\r\n" +
"* lang=CODE: 指定媒體檔案語言代碼 (非必須)\r\n" +
"* name=NAME: 指定媒體檔案描述訊息 (非必須)\r\n\r\n" +
"例如: \r\n" +
"# 引入外部字幕\r\n" +
"--mux-import path=zh-Hant.srt:lang=chi:name=\"中文 (繁體)\"\r\n" +
"# 引入外部音軌+字幕\r\n" +
"--mux-import path=\"D\\:\\media\\atmos.m4a\":lang=eng:name=\"English Description Audio\" --mux-import path=\"D\\:\\media\\eng.vtt\":lang=eng:name=\"English (Description)\"",
enUS: "When MuxAfterDone enabled, allow to import local media files. OPTIONS is a colon separated list of:\r\n\r\n" +
"* path=PATH: set file path\r\n" +
"* lang=CODE: set media language code (not required)\r\n" +
"* name=NAME: set description (not required)\r\n\r\n" +
"Examples: \r\n" +
"# import subtitle\r\n" +
"--mux-import path=en-US.srt:lang=eng:name=\"English (Original)\"\r\n" +
"# import audio and subtitle\r\n" +
"--mux-import path=\"D\\:\\media\\atmos.m4a\":lang=eng:name=\"English Description Audio\" --mux-import path=\"D\\:\\media\\eng.vtt\":lang=eng:name=\"English (Description)\""
),
["cmd_writeMetaJson"] = new TextContainer
(
zhCN: "解析后的信息是否输出json文件",
zhTW: "解析後的訊息是否輸出json文件",
enUS: "Write meta json after parsed"
),
["liveLimit"] = new TextContainer
(
zhCN: "本次直播录制时长上限: ",
zhTW: "本次直播錄製時長上限: ",
enUS: "Live recording duration limit: "
),
["realTimeDecMessage"] = new TextContainer
(
zhCN: "启用实时解密时，建议用shaka-packager而非mp4decrypt/ffmpeg",
zhTW: "啟用即時解密時，建議用shaka-packager而非mp4decrypt/ffmpeg",
enUS: "When enabling real-time decryption, it is recommended to use shaka-packager instead of mp4decrypt/ffmpeg"
),
["liveLimitReached"] = new TextContainer
(
zhCN: "到达直播录制上限，即将停止录制",
zhTW: "到達直播錄製上限，即將停止錄製",
enUS: "Live recording limit reached, will stop recording soon"
),
["saveName"] = new TextContainer
(
zhCN: "保存文件名: ",
zhTW: "保存檔案名: ",
enUS: "Save Name: "
),
["fetch"] = new TextContainer
(
zhCN: "获取: ",
zhTW: "獲取: ",
enUS: "Fetch: "
),
["ffmpegMerge"] = new TextContainer
(
zhCN: "调用ffmpeg合并中...",
zhTW: "調用ffmpeg合併中...",
enUS: "ffmpeg merging..."
),
["ffmpegNotFound"] = new TextContainer
(
zhCN: "找不到ffmpeg，请自行下载：https://ffmpeg.org/download.html",
zhTW: "找不到ffmpeg，請自行下載：https://ffmpeg.org/download.html",
enUS: "ffmpeg not found, please download at: https://ffmpeg.org/download.html"
),
["mkvmergeNotFound"] = new TextContainer
(
zhCN: "找不到mkvmerge，请自行下载：https://mkvtoolnix.download/downloads.html",
zhTW: "找不到mkvmerge，請自行下載：https://mkvtoolnix.download/downloads.html",
enUS: "mkvmerge not found, please download at: https://mkvtoolnix.download/downloads.html"
),
["shakaPackagerNotFound"] = new TextContainer
(
zhCN: "找不到shaka-packager，请自行下载：https://github.com/shaka-project/shaka-packager/releases",
zhTW: "找不到shaka-packager，請自行下載：https://github.com/shaka-project/shaka-packager/releases",
enUS: "shaka-packager not found, please download at: https://github.com/shaka-project/shaka-packager/releases"
),
["mp4decryptNotFound"] = new TextContainer
(
zhCN: "找不到mp4decrypt，请自行下载：https://www.bento4.com/downloads/",
zhTW: "找不到mp4decrypt，請自行下載：https://www.bento4.com/downloads/",
enUS: "mp4decrypt not found, please download at: https://www.bento4.com/downloads/"
),
["fixingTTML"] = new TextContainer
(
zhCN: "正在提取TTML(raw)字幕...",
zhTW: "正在提取TTML(raw)字幕...",
enUS: "Extracting TTML(raw) subtitle..."
),
["fixingTTMLmp4"] = new TextContainer
(
zhCN: "正在提取TTML(mp4)字幕...",
zhTW: "正在提取TTML(mp4)字幕...",
enUS: "Extracting TTML(mp4) subtitle..."
),
["fixingVTT"] = new TextContainer
(
zhCN: "正在提取VTT(raw)字幕...",
zhTW: "正在提取VTT(raw)字幕...",
enUS: "Extracting VTT(raw) subtitle..."
),
["fixingVTTmp4"] = new TextContainer
(
zhCN: "正在提取VTT(mp4)字幕...",
zhTW: "正在提取VTT(mp4)字幕...",
enUS: "Extracting VTT(mp4) subtitle..."
),
["keyProcessorNotFound"] = new TextContainer
(
zhCN: "找不到支持的Processor",
zhTW: "找不到支持的Processor",
enUS: "No Processor matched"
),
["liveFound"] = new TextContainer
(
zhCN: "检测到直播流",
zhTW: "檢測到直播流",
enUS: "Live stream found"
),
["loadingUrl"] = new TextContainer
(
zhCN: "加载URL: ",
zhTW: "載入URL: ",
enUS: "Loading URL: "
),
["masterM3u8Found"] = new TextContainer
(
zhCN: "检测到Master列表，开始解析全部流信息",
zhTW: "檢測到Master列表，開始解析全部流訊息",
enUS: "Master List detected, try parse all streams"
),
["allowHlsMultiExtMap"] = new TextContainer
(
zhCN: "已经允许识别多个#EXT-X-MAP标签, 本软件可能无法正确处理, 请手动确认内容完整性",
zhTW: "已經允許識別多個#EXT-X-MAP標籤, 本軟件可能無法正確處理, 請手動確認內容完整性",
enUS: "Multiple #EXT-X-MAP tags are now allowed for detection. However, this software may not handle them correctly. Please manually verify the content's integrity"
),
["matchTS"] = new TextContainer
(
zhCN: "内容匹配: [white on green3]HTTP Live MPEG2-TS[/]",
zhTW: "內容匹配: [white on green3]HTTP Live MPEG2-TS[/]",
enUS: "Content Matched: [white on green3]HTTP Live MPEG2-TS[/]"
),
["matchDASH"] = new TextContainer
(
zhCN: "内容匹配: [white on mediumorchid1]Dynamic Adaptive Streaming over HTTP[/]",
zhTW: "內容匹配: [white on mediumorchid1]Dynamic Adaptive Streaming over HTTP[/]",
enUS: "Content Matched: [white on mediumorchid1]Dynamic Adaptive Streaming over HTTP[/]"
),
["matchMSS"] = new TextContainer
(
zhCN: "内容匹配: [white on steelblue1]Microsoft Smooth Streaming[/]",
zhTW: "內容匹配: [white on steelblue1]Microsoft Smooth Streaming[/]",
enUS: "Content Matched: [white on steelblue1]Microsoft Smooth Streaming[/]"
),
["matchHLS"] = new TextContainer
(
zhCN: "内容匹配: [white on deepskyblue1]HTTP Live Streaming[/]",
zhTW: "內容匹配: [white on deepskyblue1]HTTP Live Streaming[/]",
enUS: "Content Matched: [white on deepskyblue1]HTTP Live Streaming[/]"
),
["partMerge"] = new TextContainer
(
zhCN: "分片数量大于1800个，开始分块合并...",
zhTW: "分片數量大於1800個，開始分塊合併...",
enUS: "Segments more than 1800, start partial merge..."
),
["notSupported"] = new TextContainer
(
zhCN: "当前输入不受支持: ",
zhTW: "當前輸入不受支援: ",
enUS: "Input not supported: "
),
["parsingStream"] = new TextContainer
(
zhCN: "正在解析媒体信息...",
zhTW: "正在解析媒體信息...",
enUS: "Parsing streams..."
),
["promptChoiceText"] = new TextContainer
(
zhCN: "[grey](按键盘上下键以浏览更多内容)[/]",
zhTW: "[grey](按鍵盤上下鍵以瀏覽更多內容)[/]",
enUS: "[grey](Move up and down to reveal more streams)[/]"
),
["promptInfo"] = new TextContainer
(
zhCN: "(按 [blue]空格键[/] 选择流, [green]回车键[/] 完成选择)",
zhTW: "(按 [blue]空格鍵[/] 選擇流, [green]確認鍵[/] 完成選擇)",
enUS: "(Press [blue]<space>[/] to toggle a stream, [green]<enter>[/] to accept)"
),
["promptTitle"] = new TextContainer
(
zhCN: "请选择 [green]你要下载的内容[/]:",
zhTW: "請選擇 [green]你要下載的內容[/]:",
enUS: "Please select [green]what you want to download[/]:"
),
["readingInfo"] = new TextContainer
(
zhCN: "读取媒体信息...",
zhTW: "讀取媒體訊息...",
enUS: "Reading media info..."
),
["searchKey"] = new TextContainer
(
zhCN: "正在尝试从文本文件搜索KEY...",
zhTW: "正在嘗試從文本文件搜尋KEY...",
enUS: "Trying to search for KEY from text file..."
),
["decryptionFailed"] = new TextContainer
(
zhCN: "解密失败",
zhTW: "解密失敗",
enUS: "Decryption failed"
),
["segmentCountCheckNotPass"] = new TextContainer
(
zhCN: "分片数量校验不通过, 共{}个,已下载{}.",
zhTW: "分片數量校驗不通過, 共{}個,已下載{}.",
enUS: "Segment count check not pass, total: {}, downloaded: {}."
),
["selectedStream"] = new TextContainer
(
zhCN: "已选择的流:",
zhTW: "已選擇的流:",
enUS: "Selected streams:"
),
["startDownloading"] = new TextContainer
(
zhCN: "开始下载...",
zhTW: "開始下載...",
enUS: "Start downloading..."
),
["streamsInfo"] = new TextContainer
(
zhCN: "已解析, 共计 {} 条媒体流, 基本流 {} 条, 可选音频流 {} 条, 可选字幕流 {} 条",
zhTW: "已解析, 共計 {} 條媒體流, 基本流 {} 條, 可選音頻流 {} 條, 可選字幕流 {} 條",
enUS: "Extracted, there are {} streams, with {} basic streams, {} audio streams, {} subtitle streams"
),
["writeJson"] = new TextContainer
(
zhCN: "写出meta json",
zhTW: "寫出meta json",
enUS: "Writing meta json"
),
["noStreamsToDownload"] = new TextContainer
(
zhCN: "没有找到需要下载的流",
zhTW: "沒有找到需要下載的流",
enUS: "No stream found to download"
),
};
}

15
src/N_m3u8DL-RE.Common/Resource/TextContainer.cs Normal file
View File

@ -0,0 +1,15 @@
namespace N_m3u8DL_RE.Common.Resource;
internal class TextContainer
{
public string ZH_CN { get; }
public string ZH_TW { get; }
public string EN_US { get; }
public TextContainer(string zhCN, string zhTW, string enUS)
{
ZH_CN = zhCN;
ZH_TW = zhTW;
EN_US = enUS;
}
}

73
src/N_m3u8DL-RE.Common/Util/GlobalUtil.cs Normal file
View File

@ -0,0 +1,73 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.JsonConverter;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace N_m3u8DL_RE.Common.Util;
public static class GlobalUtil
{
private static readonly JsonSerializerOptions Options = new()
{
Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
WriteIndented = true,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
Converters = { new JsonStringEnumConverter(), new BytesBase64Converter() }
};
private static readonly JsonContext Context = new JsonContext(Options);
public static string ConvertToJson(object o)
{
if (o is StreamSpec s)
{
return JsonSerializer.Serialize(s, Context.StreamSpec);
}
if (o is IOrderedEnumerable<StreamSpec> ss)
{
return JsonSerializer.Serialize(ss, Context.IOrderedEnumerableStreamSpec);
}
if (o is List<StreamSpec> sList)
{
return JsonSerializer.Serialize(sList, Context.ListStreamSpec);
}
if (o is IEnumerable<MediaSegment> mList)
{
return JsonSerializer.Serialize(mList, Context.IEnumerableMediaSegment);
}
return "{NOT SUPPORTED}";
}
public static string FormatFileSize(double fileSize)
{
return fileSize switch
{
< 0 => throw new ArgumentOutOfRangeException(nameof(fileSize)),
>= 1024 * 1024 * 1024 => $"{fileSize / (1024 * 1024 * 1024):########0.00}GB",
>= 1024 * 1024 => $"{fileSize / (1024 * 1024):####0.00}MB",
>= 1024 => $"{fileSize / 1024:####0.00}KB",
_ => $"{fileSize:####0.00}B"
};
}
// 此函数用于格式化输出时长
public static string FormatTime(int time)
{
TimeSpan ts = new TimeSpan(0, 0, time);
string str = "";
str = (ts.Hours.ToString("00") == "00" ? "" : ts.Hours.ToString("00") + "h") + ts.Minutes.ToString("00") + "m" + ts.Seconds.ToString("00") + "s";
return str;
}
/// <summary>
/// 寻找可执行程序
/// </summary>
/// <param name="name"></param>
/// <returns></returns>
public static string? FindExecutable(string name)
{
var fileExt = OperatingSystem.IsWindows() ? ".exe" : "";
var searchPath = new[] { Environment.CurrentDirectory, Path.GetDirectoryName(Environment.ProcessPath) };
var envPath = Environment.GetEnvironmentVariable("PATH")?.Split(Path.PathSeparator) ?? [];
return searchPath.Concat(envPath).Select(p => Path.Combine(p!, name + fileExt)).FirstOrDefault(File.Exists);
}
}

138
src/N_m3u8DL-RE.Common/Util/HTTPUtil.cs Normal file
View File

@ -0,0 +1,138 @@
using System.Net;
using System.Net.Http.Headers;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
namespace N_m3u8DL_RE.Common.Util;
public static class HTTPUtil
{
public static readonly HttpClientHandler HttpClientHandler = new()
{
AllowAutoRedirect = false,
AutomaticDecompression = DecompressionMethods.All,
ServerCertificateCustomValidationCallback = (sender, cert, chain, sslPolicyErrors) => true,
MaxConnectionsPerServer = 1024,
};
public static readonly HttpClient AppHttpClient = new(HttpClientHandler)
{
Timeout = TimeSpan.FromSeconds(100),
DefaultRequestVersion = HttpVersion.Version20,
DefaultVersionPolicy = HttpVersionPolicy.RequestVersionOrHigher,
};
private static async Task<HttpResponseMessage> DoGetAsync(string url, Dictionary<string, string>? headers = null)
{
Logger.Debug(ResString.fetch + url);
using var webRequest = new HttpRequestMessage(HttpMethod.Get, url);
webRequest.Headers.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate");
webRequest.Headers.CacheControl = CacheControlHeaderValue.Parse("no-cache");
webRequest.Headers.Connection.Clear();
if (headers != null)
{
foreach (var item in headers)
{
webRequest.Headers.TryAddWithoutValidation(item.Key, item.Value);
}
}
Logger.Debug(webRequest.Headers.ToString());
// 手动处理跳转，以免自定义Headers丢失
var webResponse = await AppHttpClient.SendAsync(webRequest, HttpCompletionOption.ResponseHeadersRead);
if (((int)webResponse.StatusCode).ToString().StartsWith("30"))
{
HttpResponseHeaders respHeaders = webResponse.Headers;
Logger.Debug(respHeaders.ToString());
if (respHeaders.Location != null)
{
var redirectedUrl = "";
if (!respHeaders.Location.IsAbsoluteUri)
{
Uri uri1 = new Uri(url);
Uri uri2 = new Uri(uri1, respHeaders.Location);
redirectedUrl = uri2.ToString();
}
else
{
redirectedUrl = respHeaders.Location.AbsoluteUri;
}
if (redirectedUrl != url)
{
Logger.Extra($"Redirected => {redirectedUrl}");
return await DoGetAsync(redirectedUrl, headers);
}
}
}
// 手动将跳转后的URL设置进去, 用于后续取用
webResponse.Headers.Location = new Uri(url);
webResponse.EnsureSuccessStatusCode();
return webResponse;
}
public static async Task<byte[]> GetBytesAsync(string url, Dictionary<string, string>? headers = null)
{
if (url.StartsWith("file:"))
{
return await File.ReadAllBytesAsync(new Uri(url).LocalPath);
}
var webResponse = await DoGetAsync(url, headers);
var bytes = await webResponse.Content.ReadAsByteArrayAsync();
Logger.Debug(HexUtil.BytesToHex(bytes, " "));
return bytes;
}
/// <summary>
/// 获取网页源码
/// </summary>
/// <param name="url"></param>
/// <param name="headers"></param>
/// <returns></returns>
public static async Task<string> GetWebSourceAsync(string url, Dictionary<string, string>? headers = null)
{
var webResponse = await DoGetAsync(url, headers);
string htmlCode = await webResponse.Content.ReadAsStringAsync();
Logger.Debug(htmlCode);
return htmlCode;
}
private static bool CheckMPEG2TS(HttpResponseMessage? webResponse)
{
var mediaType = webResponse?.Content.Headers.ContentType?.MediaType?.ToLower();
return mediaType is "video/ts" or "video/mp2t" or "video/mpeg";
}
/// <summary>
/// 获取网页源码和跳转后的URL
/// </summary>
/// <param name="url"></param>
/// <param name="headers"></param>
/// <returns>(Source Code, RedirectedUrl)</returns>
public static async Task<(string, string)> GetWebSourceAndNewUrlAsync(string url, Dictionary<string, string>? headers = null)
{
string htmlCode;
var webResponse = await DoGetAsync(url, headers);
if (CheckMPEG2TS(webResponse))
{
htmlCode = ResString.ReLiveTs;
}
else
{
htmlCode = await webResponse.Content.ReadAsStringAsync();
}
Logger.Debug(htmlCode);
return (htmlCode, webResponse.Headers.Location != null ? webResponse.Headers.Location.AbsoluteUri : url);
}
public static async Task<string> GetPostResponseAsync(string Url, byte[] postData)
{
string htmlCode;
using HttpRequestMessage request = new(HttpMethod.Post, Url);
request.Headers.TryAddWithoutValidation("Content-Type", "application/json");
request.Headers.TryAddWithoutValidation("Content-Length", postData.Length.ToString());
request.Content = new ByteArrayContent(postData);
var webResponse = await AppHttpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
htmlCode = await webResponse.Content.ReadAsStringAsync();
return htmlCode;
}
}

39
src/N_m3u8DL-RE.Common/Util/HexUtil.cs Normal file
View File

@ -0,0 +1,39 @@
namespace N_m3u8DL_RE.Common.Util;
public static class HexUtil
{
public static string BytesToHex(byte[] data, string split = "")
{
return BitConverter.ToString(data).Replace("-", split);
}
/// <summary>
/// 判断是不是HEX字符串
/// </summary>
/// <param name="input"></param>
/// <returns></returns>
public static bool TryParseHexString(string input, out byte[]? bytes)
{
bytes = null;
input = input.ToUpper();
if (input.StartsWith("0X"))
input = input[2..];
if (input.Length % 2 != 0)
return false;
if (input.Any(c => !"0123456789ABCDEF".Contains(c)))
return false;
bytes = HexToBytes(input);
return true;
}
public static byte[] HexToBytes(string hex)
{
var hexSpan = hex.AsSpan().Trim();
if (hexSpan.StartsWith("0x") || hexSpan.StartsWith("0X"))
{
hexSpan = hexSpan[2..];
}
return Convert.FromHexString(hexSpan);
}
}

38
src/N_m3u8DL-RE.Common/Util/RetryUtil.cs Normal file
View File

@ -0,0 +1,38 @@
using System.Net;
using N_m3u8DL_RE.Common.Log;
using Spectre.Console;
namespace N_m3u8DL_RE.Common.Util;
public static class RetryUtil
{
public static async Task<T?> WebRequestRetryAsync<T>(Func<Task<T>> funcAsync, int maxRetries = 10, int retryDelayMilliseconds = 1500, int retryDelayIncrementMilliseconds = 0)
{
var retryCount = 0;
var result = default(T);
Exception currentException = new();
while (retryCount < maxRetries)
{
try
{
result = await funcAsync();
break;
}
catch (Exception ex) when (ex is WebException or IOException or HttpRequestException)
{
currentException = ex;
retryCount++;
Logger.WarnMarkUp($"[grey]{ex.Message.EscapeMarkup()} ({retryCount}/{maxRetries})[/]");
await Task.Delay(retryDelayMilliseconds + (retryDelayIncrementMilliseconds * (retryCount - 1)));
}
}
if (retryCount == maxRetries)
{
throw new Exception($"Failed to execute action after {maxRetries} retries.", currentException);
}
return result;
}
}

69
src/N_m3u8DL-RE.Parser/Config/ParserConfig.cs Normal file
View File

@ -0,0 +1,69 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Parser.Processor;
using N_m3u8DL_RE.Parser.Processor.DASH;
using N_m3u8DL_RE.Parser.Processor.HLS;
namespace N_m3u8DL_RE.Parser.Config;
public class ParserConfig
{
public string Url { get; set; } = string.Empty;
public string OriginalUrl { get; set; } = string.Empty;
public string BaseUrl { get; set; } = string.Empty;
public Dictionary<string, string> CustomParserArgs { get; } = new();
public Dictionary<string, string> Headers { get; init; } = new();
/// <summary>
/// 内容前置处理器. 调用顺序与列表顺序相同
/// </summary>
public IList<ContentProcessor> ContentProcessors { get; } = new List<ContentProcessor>() { new DefaultHLSContentProcessor(), new DefaultDASHContentProcessor() };
/// <summary>
/// 添加分片URL前置处理器. 调用顺序与列表顺序相同
/// </summary>
public IList<UrlProcessor> UrlProcessors { get; } = new List<UrlProcessor>() { new DefaultUrlProcessor() };
/// <summary>
/// KEY解析器. 调用顺序与列表顺序相同
/// </summary>
public IList<KeyProcessor> KeyProcessors { get; } = new List<KeyProcessor>() { new DefaultHLSKeyProcessor() };
/// <summary>
/// 自定义的加密方式
/// </summary>
public EncryptMethod? CustomMethod { get; set; }
/// <summary>
/// 自定义的解密KEY
/// </summary>
public byte[]? CustomeKey { get; set; }
/// <summary>
/// 自定义的解密IV
/// </summary>
public byte[]? CustomeIV { get; set; }
/// <summary>
/// 组装视频分段的URL时，是否要把原本URL后的参数也加上去
/// 如 Base URL = "http://xxx.com/playlist.m3u8?hmac=xxx&token=xxx"
/// 相对路径 = clip_01.ts
/// 如果 AppendUrlParams=false，得 http://xxx.com/clip_01.ts
/// 如果 AppendUrlParams=true，得 http://xxx.com/clip_01.ts?hmac=xxx&token=xxx
/// </summary>
public bool AppendUrlParams { get; set; } = false;
/// <summary>
/// 此参数将会传递给URL Processor中
/// </summary>
public string? UrlProcessorArgs { get; set; }
/// <summary>
/// KEY重试次数
/// </summary>
public int KeyRetryCount { get; set; } = 3;
}

9
src/N_m3u8DL-RE.Parser/Constants/DASHTags.cs Normal file
View File

@ -0,0 +1,9 @@
namespace N_m3u8DL_RE.Parser.Constants;
internal static class DASHTags
{
public const string TemplateRepresentationID = "$RepresentationID$";
public const string TemplateBandwidth = "$Bandwidth$";
public const string TemplateNumber = "$Number$";
public const string TemplateTime = "$Time$";
}

31
src/N_m3u8DL-RE.Parser/Constants/HLSTags.cs Normal file
View File

@ -0,0 +1,31 @@
namespace N_m3u8DL_RE.Parser.Constants;
internal static class HLSTags
{
public const string ext_m3u = "#EXTM3U";
public const string ext_x_targetduration = "#EXT-X-TARGETDURATION";
public const string ext_x_media_sequence = "#EXT-X-MEDIA-SEQUENCE";
public const string ext_x_discontinuity_sequence = "#EXT-X-DISCONTINUITY-SEQUENCE";
public const string ext_x_program_date_time = "#EXT-X-PROGRAM-DATE-TIME";
public const string ext_x_media = "#EXT-X-MEDIA";
public const string ext_x_playlist_type = "#EXT-X-PLAYLIST-TYPE";
public const string ext_x_key = "#EXT-X-KEY";
public const string ext_x_stream_inf = "#EXT-X-STREAM-INF";
public const string ext_x_version = "#EXT-X-VERSION";
public const string ext_x_allow_cache = "#EXT-X-ALLOW-CACHE";
public const string ext_x_endlist = "#EXT-X-ENDLIST";
public const string extinf = "#EXTINF";
public const string ext_i_frames_only = "#EXT-X-I-FRAMES-ONLY";
public const string ext_x_byterange = "#EXT-X-BYTERANGE";
public const string ext_x_i_frame_stream_inf = "#EXT-X-I-FRAME-STREAM-INF";
public const string ext_x_discontinuity = "#EXT-X-DISCONTINUITY";
public const string ext_x_cue_out_start = "#EXT-X-CUE-OUT";
public const string ext_x_cue_out = "#EXT-X-CUE-OUT-CONT";
public const string ext_is_independent_segments = "#EXT-X-INDEPENDENT-SEGMENTS";
public const string ext_x_scte35 = "#EXT-OATCLS-SCTE35";
public const string ext_x_cue_start = "#EXT-X-CUE-OUT";
public const string ext_x_cue_end = "#EXT-X-CUE-IN";
public const string ext_x_cue_span = "#EXT-X-CUE-SPAN";
public const string ext_x_map = "#EXT-X-MAP";
public const string ext_x_start = "#EXT-X-START";
}

9
src/N_m3u8DL-RE.Parser/Constants/MSSTags.cs Normal file
View File

@ -0,0 +1,9 @@
namespace N_m3u8DL_RE.Parser.Constants;
internal static class MSSTags
{
public const string Bitrate = "{Bitrate}";
public const string Bitrate_BK = "{bitrate}";
public const string StartTime = "{start_time}";
public const string StartTime_BK = "{start time}";
}

639
src/N_m3u8DL-RE.Parser/Extractor/DASHExtractor2.cs Normal file
View File

@ -0,0 +1,639 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Parser.Constants;
using N_m3u8DL_RE.Parser.Util;
using System.Text.RegularExpressions;
using System.Xml;
using System.Xml.Linq;
namespace N_m3u8DL_RE.Parser.Extractor;
// https://blog.csdn.net/leek5533/article/details/117750191
internal partial class DASHExtractor2 : IExtractor
{
private static EncryptMethod DEFAULT_METHOD = EncryptMethod.CENC;
public ExtractorType ExtractorType => ExtractorType.MPEG_DASH;
private string MpdUrl = string.Empty;
private string BaseUrl = string.Empty;
private string MpdContent = string.Empty;
public ParserConfig ParserConfig { get; set; }
public DASHExtractor2(ParserConfig parserConfig)
{
this.ParserConfig = parserConfig;
SetInitUrl();
}
private void SetInitUrl()
{
this.MpdUrl = ParserConfig.Url ?? string.Empty;
this.BaseUrl = !string.IsNullOrEmpty(ParserConfig.BaseUrl) ? ParserConfig.BaseUrl : this.MpdUrl;
}
private string ExtendBaseUrl(XElement element, string oriBaseUrl)
{
var target = element.Elements().FirstOrDefault(e => e.Name.LocalName == "BaseURL");
if (target != null)
{
oriBaseUrl = ParserUtil.CombineURL(oriBaseUrl, target.Value);
}
return oriBaseUrl;
}
private double? GetFrameRate(XElement element)
{
var frameRate = element.Attribute("frameRate")?.Value;
if (frameRate == null || !frameRate.Contains('/')) return null;
var d = Convert.ToDouble(frameRate.Split('/')[0]) / Convert.ToDouble(frameRate.Split('/')[1]);
frameRate = d.ToString("0.000");
return Convert.ToDouble(frameRate);
}
public Task<List<StreamSpec>> ExtractStreamsAsync(string rawText)
{
var streamList = new List<StreamSpec>();
this.MpdContent = rawText;
this.PreProcessContent();
var xmlDocument = XDocument.Parse(MpdContent);
// 选中第一个MPD节点
var mpdElement = xmlDocument.Elements().First(e => e.Name.LocalName == "MPD");
// 类型 static点播, dynamic直播
var type = mpdElement.Attribute("type")?.Value;
bool isLive = type == "dynamic";
// 分片最大时长
var maxSegmentDuration = mpdElement.Attribute("maxSegmentDuration")?.Value;
// 分片从该时间起可用
var availabilityStartTime = mpdElement.Attribute("availabilityStartTime")?.Value;
// 在availabilityStartTime的前XX段时间，分片有效
var timeShiftBufferDepth = mpdElement.Attribute("timeShiftBufferDepth")?.Value;
if (string.IsNullOrEmpty(timeShiftBufferDepth))
{
// 如果没有 默认一分钟有效
timeShiftBufferDepth = "PT1M";
}
// MPD发布时间
var publishTime = mpdElement.Attribute("publishTime")?.Value;
// MPD总时长
var mediaPresentationDuration = mpdElement.Attribute("mediaPresentationDuration")?.Value;
// 读取在MPD开头定义的<BaseURL>，并替换本身的URL
var baseUrlElement = mpdElement.Elements().FirstOrDefault(e => e.Name.LocalName == "BaseURL");
if (baseUrlElement != null)
{
var baseUrl = baseUrlElement.Value;
if (baseUrl.Contains("kkbox.com.tw/")) baseUrl = baseUrl.Replace("//https:%2F%2F", "//");
this.BaseUrl = ParserUtil.CombineURL(this.MpdUrl, baseUrl);
}
// 全部Period
var periods = mpdElement.Elements().Where(e => e.Name.LocalName == "Period");
foreach (var period in periods)
{
// 本Period时长
var periodDuration = period.Attribute("duration")?.Value;
// 本Period ID
var periodId = period.Attribute("id")?.Value;
// 最终分片会使用的baseurl
var segBaseUrl = this.BaseUrl;
// 处理baseurl嵌套
segBaseUrl = ExtendBaseUrl(period, segBaseUrl);
var adaptationSetsBaseUrl = segBaseUrl;
// 本Period中的全部AdaptationSet
var adaptationSets = period.Elements().Where(e => e.Name.LocalName == "AdaptationSet");
foreach (var adaptationSet in adaptationSets)
{
// 处理baseurl嵌套
segBaseUrl = ExtendBaseUrl(adaptationSet, segBaseUrl);
var representationsBaseUrl = segBaseUrl;
var mimeType = adaptationSet.Attribute("contentType")?.Value ?? adaptationSet.Attribute("mimeType")?.Value;
var frameRate = GetFrameRate(adaptationSet);
// 本AdaptationSet中的全部Representation
var representations = adaptationSet.Elements().Where(e => e.Name.LocalName == "Representation");
foreach (var representation in representations)
{
// 处理baseurl嵌套
segBaseUrl = ExtendBaseUrl(representation, segBaseUrl);
if (mimeType == null)
{
mimeType = representation.Attribute("contentType")?.Value ?? representation.Attribute("mimeType")?.Value ?? "";
}
var bandwidth = representation.Attribute("bandwidth");
StreamSpec streamSpec = new();
streamSpec.OriginalUrl = ParserConfig.OriginalUrl;
streamSpec.PeriodId = periodId;
streamSpec.Playlist = new Playlist();
streamSpec.Playlist.MediaParts.Add(new MediaPart());
streamSpec.GroupId = representation.Attribute("id")?.Value;
streamSpec.Bandwidth = Convert.ToInt32(bandwidth?.Value ?? "0");
streamSpec.Codecs = representation.Attribute("codecs")?.Value ?? adaptationSet.Attribute("codecs")?.Value;
streamSpec.Language = FilterLanguage(representation.Attribute("lang")?.Value ?? adaptationSet.Attribute("lang")?.Value);
streamSpec.FrameRate = frameRate ?? GetFrameRate(representation);
streamSpec.Resolution = representation.Attribute("width")?.Value != null ? $"{representation.Attribute("width")?.Value}x{representation.Attribute("height")?.Value}" : null;
streamSpec.Url = MpdUrl;
streamSpec.MediaType = mimeType.Split('/')[0] switch
{
"text" => MediaType.SUBTITLES,
"audio" => MediaType.AUDIO,
_ => null
};
// 特殊处理
if (representation.Attribute("volumeAdjust") != null)
{
streamSpec.GroupId += "-" + representation.Attribute("volumeAdjust")?.Value;
}
// 推测后缀名
var mType = representation.Attribute("mimeType")?.Value ?? adaptationSet.Attribute("mimeType")?.Value;
if (mType != null)
{
var mTypeSplit = mType.Split('/');
streamSpec.Extension = mTypeSplit.Length == 2 ? mTypeSplit[1] : null;
}
// 优化字幕场景识别
if (streamSpec.Codecs is "stpp" or "wvtt")
{
streamSpec.MediaType = MediaType.SUBTITLES;
}
// 优化字幕场景识别
var role = representation.Elements().FirstOrDefault(e => e.Name.LocalName == "Role") ?? adaptationSet.Elements().FirstOrDefault(e => e.Name.LocalName == "Role");
if (role != null)
{
var roleValue = role.Attribute("value")?.Value;
if (Enum.TryParse(roleValue, true, out RoleType roleType))
{
streamSpec.Role = roleType;
if (roleType == RoleType.Subtitle)
{
streamSpec.MediaType = MediaType.SUBTITLES;
if (mType != null && mType.Contains("ttml"))
streamSpec.Extension = "ttml";
}
}
else if (roleValue != null && roleValue.Contains('-'))
{
roleValue = roleValue.Replace("-", "");
if (Enum.TryParse(roleValue, true, out RoleType roleType_))
{
streamSpec.Role = roleType_;
if (roleType_ == RoleType.ForcedSubtitle)
{
streamSpec.MediaType = MediaType.SUBTITLES; // or maybe MediaType.CLOSED_CAPTIONS?
if (mType != null && mType.Contains("ttml"))
streamSpec.Extension = "ttml";
}
}
}
}
streamSpec.Playlist.IsLive = isLive;
// 设置刷新间隔 timeShiftBufferDepth / 2
if (timeShiftBufferDepth != null)
{
streamSpec.Playlist.RefreshIntervalMs = XmlConvert.ToTimeSpan(timeShiftBufferDepth).TotalMilliseconds / 2;
}
// 读取声道数量
var audioChannelConfiguration = adaptationSet.Elements().Concat(representation.Elements()).FirstOrDefault(e => e.Name.LocalName == "AudioChannelConfiguration");
if (audioChannelConfiguration != null)
{
streamSpec.Channels = audioChannelConfiguration.Attribute("value")?.Value;
}
// 发布时间
if (!string.IsNullOrEmpty(publishTime))
{
streamSpec.PublishTime = DateTime.Parse(publishTime);
}
// 第一种形式 SegmentBase
var segmentBaseElement = representation.Elements().FirstOrDefault(e => e.Name.LocalName == "SegmentBase");
if (segmentBaseElement != null)
{
// 处理init url
var initialization = segmentBaseElement.Elements().FirstOrDefault(e => e.Name.LocalName == "Initialization");
if (initialization != null)
{
var sourceURL = initialization.Attribute("sourceURL")?.Value;
if (sourceURL == null)
{
streamSpec.Playlist.MediaParts[0].MediaSegments.Add
(
new MediaSegment()
{
Index = 0,
Url = segBaseUrl,
Duration = XmlConvert.ToTimeSpan(periodDuration ?? mediaPresentationDuration ?? "PT0S").TotalSeconds
}
);
}
else
{
var initUrl = ParserUtil.CombineURL(segBaseUrl, initialization.Attribute("sourceURL")?.Value!);
var initRange = initialization.Attribute("range")?.Value;
streamSpec.Playlist.MediaInit = new MediaSegment();
streamSpec.Playlist.MediaInit.Index = -1; // 便于排序
streamSpec.Playlist.MediaInit.Url = initUrl;
if (initRange != null)
{
var (start, expect) = ParserUtil.ParseRange(initRange);
streamSpec.Playlist.MediaInit.StartRange = start;
streamSpec.Playlist.MediaInit.ExpectLength = expect;
}
}
}
}
// 第二种形式 SegmentList.SegmentList
var segmentList = representation.Elements().FirstOrDefault(e => e.Name.LocalName == "SegmentList");
if (segmentList != null)
{
var durationStr = segmentList.Attribute("duration")?.Value;
// 处理init url
var initialization = segmentList.Elements().FirstOrDefault(e => e.Name.LocalName == "Initialization");
if (initialization != null)
{
var initUrl = ParserUtil.CombineURL(segBaseUrl, initialization.Attribute("sourceURL")?.Value!);
var initRange = initialization.Attribute("range")?.Value;
streamSpec.Playlist.MediaInit = new MediaSegment();
streamSpec.Playlist.MediaInit.Index = -1; // 便于排序
streamSpec.Playlist.MediaInit.Url = initUrl;
if (initRange != null)
{
var (start, expect) = ParserUtil.ParseRange(initRange);
streamSpec.Playlist.MediaInit.StartRange = start;
streamSpec.Playlist.MediaInit.ExpectLength = expect;
}
}
// 处理分片
var segmentURLs = segmentList.Elements().Where(e => e.Name.LocalName == "SegmentURL").ToList();
var timescaleStr = segmentList.Attribute("timescale")?.Value ?? "1";
for (int segmentIndex = 0; segmentIndex < segmentURLs.Count; segmentIndex++)
{
var segmentURL = segmentURLs.ElementAt(segmentIndex);
var mediaUrl = ParserUtil.CombineURL(segBaseUrl, segmentURL.Attribute("media")?.Value!);
var mediaRange = segmentURL.Attribute("mediaRange")?.Value;
var timesacle = Convert.ToInt32(timescaleStr);
var duration = Convert.ToInt64(durationStr);
MediaSegment mediaSegment = new();
mediaSegment.Duration = duration / (double)timesacle;
mediaSegment.Url = mediaUrl;
mediaSegment.Index = segmentIndex;
if (mediaRange != null)
{
var (start, expect) = ParserUtil.ParseRange(mediaRange);
mediaSegment.StartRange = start;
mediaSegment.ExpectLength = expect;
}
streamSpec.Playlist.MediaParts[0].MediaSegments.Add(mediaSegment);
}
}
// 第三种形式 SegmentTemplate+SegmentTimeline
// 通配符有$RepresentationID$ $Bandwidth$ $Number$ $Time$
// adaptationSets中的segmentTemplate
var segmentTemplateElementsOuter = adaptationSet.Elements().Where(e => e.Name.LocalName == "SegmentTemplate");
// representation中的segmentTemplate
var segmentTemplateElements = representation.Elements().Where(e => e.Name.LocalName == "SegmentTemplate");
if (segmentTemplateElements.Any() || segmentTemplateElementsOuter.Any())
{
// 优先使用最近的元素
var segmentTemplate = (segmentTemplateElements.FirstOrDefault() ?? segmentTemplateElementsOuter.FirstOrDefault())!;
var segmentTemplateOuter = (segmentTemplateElementsOuter.FirstOrDefault() ?? segmentTemplateElements.FirstOrDefault())!;
var varDic = new Dictionary<string, object?>();
varDic[DASHTags.TemplateRepresentationID] = streamSpec.GroupId;
varDic[DASHTags.TemplateBandwidth] = bandwidth?.Value;
// presentationTimeOffset
var presentationTimeOffsetStr = segmentTemplate.Attribute("presentationTimeOffset")?.Value ?? segmentTemplateOuter.Attribute("presentationTimeOffset")?.Value ?? "0";
// timesacle
var timescaleStr = segmentTemplate.Attribute("timescale")?.Value ?? segmentTemplateOuter.Attribute("timescale")?.Value ?? "1";
var durationStr = segmentTemplate.Attribute("duration")?.Value ?? segmentTemplateOuter.Attribute("duration")?.Value;
var startNumberStr = segmentTemplate.Attribute("startNumber")?.Value ?? segmentTemplateOuter.Attribute("startNumber")?.Value ?? "1";
// 处理init url
var initialization = segmentTemplate.Attribute("initialization")?.Value ?? segmentTemplateOuter.Attribute("initialization")?.Value;
if (initialization != null)
{
var _init = ParserUtil.ReplaceVars(initialization, varDic);
var initUrl = ParserUtil.CombineURL(segBaseUrl, _init);
streamSpec.Playlist.MediaInit = new MediaSegment();
streamSpec.Playlist.MediaInit.Index = -1; // 便于排序
streamSpec.Playlist.MediaInit.Url = initUrl;
}
// 处理分片
var mediaTemplate = segmentTemplate.Attribute("media")?.Value ?? segmentTemplateOuter.Attribute("media")?.Value;
var segmentTimeline = segmentTemplate.Elements().FirstOrDefault(e => e.Name.LocalName == "SegmentTimeline");
if (segmentTimeline != null)
{
// 使用了SegmentTimeline 结果精确
var segNumber = Convert.ToInt64(startNumberStr);
var Ss = segmentTimeline.Elements().Where(e => e.Name.LocalName == "S");
var currentTime = 0L;
var segIndex = 0;
foreach (var S in Ss)
{
// 每个S元素包含三个属性:@t(start time)\@r(repeat count)\@d(duration)
var _startTimeStr = S.Attribute("t")?.Value;
var _durationStr = S.Attribute("d")?.Value;
var _repeatCountStr = S.Attribute("r")?.Value;
if (_startTimeStr != null) currentTime = Convert.ToInt64(_startTimeStr);
var _duration = Convert.ToInt64(_durationStr);
var timescale = Convert.ToInt32(timescaleStr);
var _repeatCount = Convert.ToInt64(_repeatCountStr);
varDic[DASHTags.TemplateTime] = currentTime;
varDic[DASHTags.TemplateNumber] = segNumber++;
var hasTime = mediaTemplate!.Contains(DASHTags.TemplateTime);
var media = ParserUtil.ReplaceVars(mediaTemplate!, varDic);
var mediaUrl = ParserUtil.CombineURL(segBaseUrl, media!);
MediaSegment mediaSegment = new();
mediaSegment.Url = mediaUrl;
if (hasTime)
mediaSegment.NameFromVar = currentTime.ToString();
mediaSegment.Duration = _duration / (double)timescale;
mediaSegment.Index = segIndex++;
streamSpec.Playlist.MediaParts[0].MediaSegments.Add(mediaSegment);
if (_repeatCount < 0)
{
// 负数表示一直重复 直到period结束 注意减掉已经加入的1个片段
_repeatCount = (long)Math.Ceiling(XmlConvert.ToTimeSpan(periodDuration ?? mediaPresentationDuration ?? "PT0S").TotalSeconds * timescale / _duration) - 1;
}
for (long i = 0; i < _repeatCount; i++)
{
currentTime += _duration;
MediaSegment _mediaSegment = new();
varDic[DASHTags.TemplateTime] = currentTime;
varDic[DASHTags.TemplateNumber] = segNumber++;
var _hashTime = mediaTemplate!.Contains(DASHTags.TemplateTime);
var _media = ParserUtil.ReplaceVars(mediaTemplate!, varDic);
var _mediaUrl = ParserUtil.CombineURL(segBaseUrl, _media);
_mediaSegment.Url = _mediaUrl;
_mediaSegment.Index = segIndex++;
_mediaSegment.Duration = _duration / (double)timescale;
if (_hashTime)
_mediaSegment.NameFromVar = currentTime.ToString();
streamSpec.Playlist.MediaParts[0].MediaSegments.Add(_mediaSegment);
}
currentTime += _duration;
}
}
else
{
// 没用SegmentTimeline 需要计算总分片数量 不精确
var timescale = Convert.ToInt32(timescaleStr);
var startNumber = Convert.ToInt64(startNumberStr);
var duration = Convert.ToInt32(durationStr);
var totalNumber = (long)Math.Ceiling(XmlConvert.ToTimeSpan(periodDuration ?? mediaPresentationDuration ?? "PT0S").TotalSeconds * timescale / duration);
// 直播的情况，需要自己计算totalNumber
if (totalNumber == 0 && isLive)
{
var now = DateTime.Now;
var availableTime = DateTime.Parse(availabilityStartTime!);
// 可用时间+偏移量
var offsetMs = TimeSpan.FromMilliseconds(Convert.ToInt64(presentationTimeOffsetStr) / 1000);
availableTime = availableTime.Add(offsetMs);
var ts = now - availableTime;
var updateTs = XmlConvert.ToTimeSpan(timeShiftBufferDepth!);
// (当前时间到发布时间的时间差 - 最小刷新间隔) / 分片时长
startNumber += (long)((ts.TotalSeconds - updateTs.TotalSeconds) * timescale / duration);
totalNumber = (long)(updateTs.TotalSeconds * timescale / duration);
}
for (long index = startNumber, segIndex = 0; index < startNumber + totalNumber; index++, segIndex++)
{
varDic[DASHTags.TemplateNumber] = index;
var hasNumber = mediaTemplate!.Contains(DASHTags.TemplateNumber);
var media = ParserUtil.ReplaceVars(mediaTemplate!, varDic);
var mediaUrl = ParserUtil.CombineURL(segBaseUrl, media!);
MediaSegment mediaSegment = new();
mediaSegment.Url = mediaUrl;
if (hasNumber)
mediaSegment.NameFromVar = index.ToString();
mediaSegment.Index = isLive ? index : segIndex; // 直播直接用startNumber
mediaSegment.Duration = duration / (double)timescale;
streamSpec.Playlist.MediaParts[0].MediaSegments.Add(mediaSegment);
}
}
}
// 如果依旧没被添加分片，直接把BaseUrl塞进去就好
if (streamSpec.Playlist.MediaParts[0].MediaSegments.Count == 0)
{
streamSpec.Playlist.MediaParts[0].MediaSegments.Add
(
new MediaSegment()
{
Index = 0,
Url = segBaseUrl,
Duration = XmlConvert.ToTimeSpan(periodDuration ?? mediaPresentationDuration ?? "PT0S").TotalSeconds
}
);
}
// 判断加密情况
if (adaptationSet.Elements().Concat(representation.Elements()).Any(e => e.Name.LocalName == "ContentProtection"))
{
if (streamSpec.Playlist.MediaInit != null)
{
streamSpec.Playlist.MediaInit.EncryptInfo.Method = DEFAULT_METHOD;
}
foreach (var item in streamSpec.Playlist.MediaParts[0].MediaSegments)
{
item.EncryptInfo.Method = DEFAULT_METHOD;
}
}
// 处理同一ID分散在不同Period的情况
var _index = streamList.FindIndex(_f => _f.PeriodId != streamSpec.PeriodId && _f.GroupId == streamSpec.GroupId && _f.Resolution == streamSpec.Resolution && _f.MediaType == streamSpec.MediaType);
if (_index > -1)
{
if (isLive)
{
// 直播，这种情况直接略过新的
}
else
{
// 点播，这种情况如果URL不同则作为新的part出现，否则仅把时间加起来
var url1 = streamList[_index].Playlist!.MediaParts.Last().MediaSegments.Last().Url;
var url2 = streamSpec.Playlist.MediaParts[0].MediaSegments.LastOrDefault()?.Url;
if (url1 != url2)
{
var startIndex = streamList[_index].Playlist!.MediaParts.Last().MediaSegments.Last().Index + 1;
var enumerator = streamSpec.Playlist.MediaParts[0].MediaSegments.GetEnumerator();
while (enumerator.MoveNext())
{
enumerator.Current.Index += startIndex;
}
streamList[_index].Playlist!.MediaParts.Add(new MediaPart()
{
MediaSegments = streamSpec.Playlist.MediaParts[0].MediaSegments
});
}
else
{
streamList[_index].Playlist!.MediaParts.Last().MediaSegments.Last().Duration += streamSpec.Playlist.MediaParts[0].MediaSegments.Sum(x => x.Duration);
}
}
}
else
{
// 修复mp4类型字幕
if (streamSpec is { MediaType: MediaType.SUBTITLES, Extension: "mp4" })
{
streamSpec.Extension = "m4s";
}
// 分片默认后缀m4s
if (streamSpec.MediaType != MediaType.SUBTITLES && (streamSpec.Extension == null || streamSpec.Playlist.MediaParts.Sum(x => x.MediaSegments.Count) > 1))
{
streamSpec.Extension = "m4s";
}
streamList.Add(streamSpec);
}
// 恢复BaseURL相对位置
segBaseUrl = representationsBaseUrl;
}
// 恢复BaseURL相对位置
segBaseUrl = adaptationSetsBaseUrl;
}
}
// 为视频设置默认轨道
var aL = streamList.Where(s => s.MediaType == MediaType.AUDIO).ToList();
var sL = streamList.Where(s => s.MediaType == MediaType.SUBTITLES).ToList();
foreach (var item in streamList.Where(item => !string.IsNullOrEmpty(item.Resolution)))
{
if (aL.Count != 0)
{
item.AudioId = aL.OrderByDescending(x => x.Bandwidth).First().GroupId;
}
if (sL.Count != 0)
{
item.SubtitleId = sL.OrderByDescending(x => x.Bandwidth).First().GroupId;
}
}
return Task.FromResult(streamList);
}
/// <summary>
/// 如果有非法字符 返回und
/// </summary>
/// <param name="v"></param>
/// <returns></returns>
private string? FilterLanguage(string? v)
{
if (v == null) return null;
return LangCodeRegex().IsMatch(v) ? v : "und";
}
public async Task RefreshPlayListAsync(List<StreamSpec> streamSpecs)
{
if (streamSpecs.Count == 0) return;
var (rawText, url) = ("", ParserConfig.Url);
try
{
(rawText, url) = await HTTPUtil.GetWebSourceAndNewUrlAsync(ParserConfig.Url, ParserConfig.Headers);
}
catch (HttpRequestException) when (ParserConfig.Url!= ParserConfig.OriginalUrl)
{
// 当URL无法访问时，再请求原始URL
(rawText, url) = await HTTPUtil.GetWebSourceAndNewUrlAsync(ParserConfig.OriginalUrl, ParserConfig.Headers);
}
ParserConfig.Url = url;
SetInitUrl();
var newStreams = await ExtractStreamsAsync(rawText);
foreach (var streamSpec in streamSpecs)
{
// 有的网站每次请求MPD返回的码率不一致，导致ToShortString()无法匹配 无法更新playlist
// 故增加通过init url来匹配 (如果有的话)
var match = newStreams.Where(n => n.ToShortString() == streamSpec.ToShortString());
if (!match.Any())
match = newStreams.Where(n => n.Playlist?.MediaInit?.Url == streamSpec.Playlist?.MediaInit?.Url);
if (match.Any())
streamSpec.Playlist!.MediaParts = match.First().Playlist!.MediaParts; // 不更新init
}
// 这里才调用URL预处理器，节省开销
await ProcessUrlAsync(streamSpecs);
}
private Task ProcessUrlAsync(List<StreamSpec> streamSpecs)
{
foreach (var streamSpec in streamSpecs)
{
var playlist = streamSpec.Playlist;
if (playlist == null) continue;
if (playlist.MediaInit != null)
{
playlist.MediaInit!.Url = PreProcessUrl(playlist.MediaInit!.Url);
}
for (var ii = 0; ii < playlist!.MediaParts.Count; ii++)
{
var part = playlist.MediaParts[ii];
foreach (var mediaSegment in part.MediaSegments)
{
mediaSegment.Url = PreProcessUrl(mediaSegment.Url);
}
}
}
return Task.CompletedTask;
}
public async Task FetchPlayListAsync(List<StreamSpec> streamSpecs)
{
// 这里才调用URL预处理器，节省开销
await ProcessUrlAsync(streamSpecs);
}
public string PreProcessUrl(string url)
{
foreach (var p in ParserConfig.UrlProcessors)
{
if (p.CanProcess(ExtractorType, url, ParserConfig))
{
url = p.Process(url, ParserConfig);
}
}
return url;
}
public void PreProcessContent()
{
foreach (var p in ParserConfig.ContentProcessors)
{
if (p.CanProcess(ExtractorType, MpdContent, ParserConfig))
{
MpdContent = p.Process(MpdContent, ParserConfig);
}
}
}
[GeneratedRegex(@"^[\w_\-\d]+$")]
private static partial Regex LangCodeRegex();
}

572
src/N_m3u8DL-RE.Parser/Extractor/HLSExtractor.cs Normal file
View File

@ -0,0 +1,572 @@
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Parser.Util;
using N_m3u8DL_RE.Parser.Constants;
using N_m3u8DL_RE.Common.Util;
namespace N_m3u8DL_RE.Parser.Extractor;
internal class HLSExtractor : IExtractor
{
public ExtractorType ExtractorType => ExtractorType.HLS;
private string M3u8Url = string.Empty;
private string BaseUrl = string.Empty;
private string M3u8Content = string.Empty;
private bool MasterM3u8Flag = false;
public ParserConfig ParserConfig { get; set; }
public HLSExtractor(ParserConfig parserConfig)
{
this.ParserConfig = parserConfig;
this.M3u8Url = parserConfig.Url ?? string.Empty;
this.SetBaseUrl();
}
private void SetBaseUrl()
{
this.BaseUrl = !string.IsNullOrEmpty(ParserConfig.BaseUrl) ? ParserConfig.BaseUrl : this.M3u8Url;
}
/// <summary>
/// 预处理m3u8内容
/// </summary>
public void PreProcessContent()
{
M3u8Content = M3u8Content.Trim();
if (!M3u8Content.StartsWith(HLSTags.ext_m3u))
{
throw new Exception(ResString.badM3u8);
}
foreach (var p in ParserConfig.ContentProcessors)
{
if (p.CanProcess(ExtractorType, M3u8Content, ParserConfig))
{
M3u8Content = p.Process(M3u8Content, ParserConfig);
}
}
}
/// <summary>
/// 预处理URL
/// </summary>
public string PreProcessUrl(string url)
{
foreach (var p in ParserConfig.UrlProcessors)
{
if (p.CanProcess(ExtractorType, url, ParserConfig))
{
url = p.Process(url, ParserConfig);
}
}
return url;
}
private Task<List<StreamSpec>> ParseMasterListAsync()
{
MasterM3u8Flag = true;
List<StreamSpec> streams = [];
using StringReader sr = new StringReader(M3u8Content);
string? line;
bool expectPlaylist = false;
StreamSpec streamSpec = new();
while ((line = sr.ReadLine()) != null)
{
if (string.IsNullOrEmpty(line))
continue;
if (line.StartsWith(HLSTags.ext_x_stream_inf))
{
streamSpec = new();
streamSpec.OriginalUrl = ParserConfig.OriginalUrl;
var bandwidth = string.IsNullOrEmpty(ParserUtil.GetAttribute(line, "AVERAGE-BANDWIDTH")) ? ParserUtil.GetAttribute(line, "BANDWIDTH") : ParserUtil.GetAttribute(line, "AVERAGE-BANDWIDTH");
streamSpec.Bandwidth = Convert.ToInt32(bandwidth);
streamSpec.Codecs = ParserUtil.GetAttribute(line, "CODECS");
streamSpec.Resolution = ParserUtil.GetAttribute(line, "RESOLUTION");
var frameRate = ParserUtil.GetAttribute(line, "FRAME-RATE");
if (!string.IsNullOrEmpty(frameRate))
streamSpec.FrameRate = Convert.ToDouble(frameRate);
var audioId = ParserUtil.GetAttribute(line, "AUDIO");
if (!string.IsNullOrEmpty(audioId))
streamSpec.AudioId = audioId;
var videoId = ParserUtil.GetAttribute(line, "VIDEO");
if (!string.IsNullOrEmpty(videoId))
streamSpec.VideoId = videoId;
var subtitleId = ParserUtil.GetAttribute(line, "SUBTITLES");
if (!string.IsNullOrEmpty(subtitleId))
streamSpec.SubtitleId = subtitleId;
var videoRange = ParserUtil.GetAttribute(line, "VIDEO-RANGE");
if (!string.IsNullOrEmpty(videoRange))
streamSpec.VideoRange = videoRange;
// 清除多余的编码信息 dvh1.05.06,ec-3 => dvh1.05.06
if (!string.IsNullOrEmpty(streamSpec.Codecs) && !string.IsNullOrEmpty(streamSpec.AudioId))
{
streamSpec.Codecs = streamSpec.Codecs.Split(',')[0];
}
expectPlaylist = true;
}
else if (line.StartsWith(HLSTags.ext_x_media))
{
streamSpec = new();
var type = ParserUtil.GetAttribute(line, "TYPE").Replace("-", "_");
if (Enum.TryParse<MediaType>(type, out var mediaType))
{
streamSpec.MediaType = mediaType;
}
// 跳过CLOSED_CAPTIONS类型（目前不支持）
if (streamSpec.MediaType == MediaType.CLOSED_CAPTIONS)
{
continue;
}
var url = ParserUtil.GetAttribute(line, "URI");
/**
* The URI attribute of the EXT-X-MEDIA tag is REQUIRED if the media
type is SUBTITLES, but OPTIONAL if the media type is VIDEO or AUDIO.
If the media type is VIDEO or AUDIO, a missing URI attribute
indicates that the media data for this Rendition is included in the
Media Playlist of any EXT-X-STREAM-INF tag referencing this EXT-
X-MEDIA tag. If the media TYPE is AUDIO and the URI attribute is
missing, clients MUST assume that the audio data for this Rendition
is present in every video Rendition specified by the EXT-X-STREAM-INF
tag.
此处直接忽略URI属性为空的情况
*/
if (string.IsNullOrEmpty(url))
{
continue;
}
url = ParserUtil.CombineURL(BaseUrl, url);
streamSpec.Url = PreProcessUrl(url);
var groupId = ParserUtil.GetAttribute(line, "GROUP-ID");
streamSpec.GroupId = groupId;
var lang = ParserUtil.GetAttribute(line, "LANGUAGE");
if (!string.IsNullOrEmpty(lang))
streamSpec.Language = lang;
var name = ParserUtil.GetAttribute(line, "NAME");
if (!string.IsNullOrEmpty(name))
streamSpec.Name = name;
var def = ParserUtil.GetAttribute(line, "DEFAULT");
if (Enum.TryParse<Choise>(type, out var defaultChoise))
{
streamSpec.Default = defaultChoise;
}
var channels = ParserUtil.GetAttribute(line, "CHANNELS");
if (!string.IsNullOrEmpty(channels))
streamSpec.Channels = channels;
var characteristics = ParserUtil.GetAttribute(line, "CHARACTERISTICS");
if (!string.IsNullOrEmpty(characteristics))
streamSpec.Characteristics = characteristics.Split(',').Last().Split('.').Last();
streams.Add(streamSpec);
}
else if (line.StartsWith('#'))
{
continue;
}
else if (expectPlaylist)
{
var url = ParserUtil.CombineURL(BaseUrl, line);
streamSpec.Url = PreProcessUrl(url);
expectPlaylist = false;
streams.Add(streamSpec);
}
}
return Task.FromResult(streams);
}
private Task<Playlist> ParseListAsync()
{
// 标记是否已清除广告分片
bool hasAd = false;
;
bool allowHlsMultiExtMap = ParserConfig.CustomParserArgs.TryGetValue("AllowHlsMultiExtMap", out var allMultiExtMap) && allMultiExtMap == "true";
if (allowHlsMultiExtMap)
{
Logger.WarnMarkUp($"[darkorange3_1]{ResString.allowHlsMultiExtMap}[/]");
}
using StringReader sr = new StringReader(M3u8Content);
string? line;
bool expectSegment = false;
bool isEndlist = false;
long segIndex = 0;
bool isAd = false;
long startIndex;
Playlist playlist = new();
List<MediaPart> mediaParts = [];
// 当前的加密信息
EncryptInfo currentEncryptInfo = new();
if (ParserConfig.CustomMethod != null)
currentEncryptInfo.Method = ParserConfig.CustomMethod.Value;
if (ParserConfig.CustomeKey is { Length: > 0 })
currentEncryptInfo.Key = ParserConfig.CustomeKey;
if (ParserConfig.CustomeIV is { Length: > 0 })
currentEncryptInfo.IV = ParserConfig.CustomeIV;
// 上次读取到的加密行，#EXT-X-KEY:……
string lastKeyLine = "";
MediaPart mediaPart = new();
MediaSegment segment = new();
List<MediaSegment> segments = [];
while ((line = sr.ReadLine()) != null)
{
if (string.IsNullOrEmpty(line))
continue;
// 只下载部分字节
if (line.StartsWith(HLSTags.ext_x_byterange))
{
var p = ParserUtil.GetAttribute(line);
var (n, o) = ParserUtil.GetRange(p);
segment.ExpectLength = n;
segment.StartRange = o ?? segments.Last().StartRange + segments.Last().ExpectLength;
expectSegment = true;
}
// 国家地理去广告
else if (line.StartsWith("#UPLYNK-SEGMENT"))
{
if (line.Contains(",ad"))
isAd = true;
else if (line.Contains(",segment"))
isAd = false;
}
// 国家地理去广告
else if (isAd)
{
continue;
}
// 解析定义的分段长度
else if (line.StartsWith(HLSTags.ext_x_targetduration))
{
playlist.TargetDuration = Convert.ToDouble(ParserUtil.GetAttribute(line));
}
// 解析起始编号
else if (line.StartsWith(HLSTags.ext_x_media_sequence))
{
segIndex = Convert.ToInt64(ParserUtil.GetAttribute(line));
startIndex = segIndex;
}
// program date time
else if (line.StartsWith(HLSTags.ext_x_program_date_time))
{
segment.DateTime = DateTime.Parse(ParserUtil.GetAttribute(line));
}
// 解析不连续标记，需要单独合并（timestamp不同）
else if (line.StartsWith(HLSTags.ext_x_discontinuity))
{
// 修复YK去除广告后的遗留问题
if (hasAd && mediaParts.Count > 0)
{
segments = mediaParts[^1].MediaSegments;
mediaParts.RemoveAt(mediaParts.Count - 1);
hasAd = false;
continue;
}
// 常规情况的#EXT-X-DISCONTINUITY标记，新建part
if (hasAd || segments.Count < 1) continue;
mediaParts.Add(new MediaPart
{
MediaSegments = segments,
});
segments = new();
}
// 解析KEY
else if (line.StartsWith(HLSTags.ext_x_key))
{
var uri = ParserUtil.GetAttribute(line, "URI");
var uri_last = ParserUtil.GetAttribute(lastKeyLine, "URI");
// 如果KEY URL相同，不进行重复解析
if (uri != uri_last)
{
// 调用处理器进行解析
var parsedInfo = ParseKey(line);
currentEncryptInfo.Method = parsedInfo.Method;
currentEncryptInfo.Key = parsedInfo.Key;
currentEncryptInfo.IV = parsedInfo.IV;
}
lastKeyLine = line;
}
// 解析分片时长
else if (line.StartsWith(HLSTags.extinf))
{
string[] tmp = ParserUtil.GetAttribute(line).Split(',');
segment.Duration = Convert.ToDouble(tmp[0]);
segment.Index = segIndex;
// 是否有加密，有的话写入KEY和IV
if (currentEncryptInfo.Method != EncryptMethod.NONE)
{
segment.EncryptInfo.Method = currentEncryptInfo.Method;
segment.EncryptInfo.Key = currentEncryptInfo.Key;
segment.EncryptInfo.IV = currentEncryptInfo.IV ?? HexUtil.HexToBytes(Convert.ToString(segIndex, 16).PadLeft(32, '0'));
}
expectSegment = true;
segIndex++;
}
// m3u8主体结束
else if (line.StartsWith(HLSTags.ext_x_endlist))
{
if (segments.Count > 0)
{
mediaParts.Add(new MediaPart()
{
MediaSegments = segments
});
}
segments = new();
isEndlist = true;
}
// #EXT-X-MAP
else if (line.StartsWith(HLSTags.ext_x_map))
{
if (playlist.MediaInit == null || hasAd)
{
playlist.MediaInit = new MediaSegment()
{
Url = PreProcessUrl(ParserUtil.CombineURL(BaseUrl, ParserUtil.GetAttribute(line, "URI"))),
Index = -1, // 便于排序
};
if (line.Contains("BYTERANGE"))
{
var p = ParserUtil.GetAttribute(line, "BYTERANGE");
var (n, o) = ParserUtil.GetRange(p);
playlist.MediaInit.ExpectLength = n;
playlist.MediaInit.StartRange = o ?? 0L;
}
if (currentEncryptInfo.Method == EncryptMethod.NONE) continue;
// 有加密的话写入KEY和IV
playlist.MediaInit.EncryptInfo.Method = currentEncryptInfo.Method;
playlist.MediaInit.EncryptInfo.Key = currentEncryptInfo.Key;
playlist.MediaInit.EncryptInfo.IV = currentEncryptInfo.IV ?? HexUtil.HexToBytes(Convert.ToString(segIndex, 16).PadLeft(32, '0'));
}
// 遇到了其他的map，说明已经不是一个视频了，全部丢弃即可
else
{
if (segments.Count > 0)
{
mediaParts.Add(new MediaPart()
{
MediaSegments = segments
});
}
segments = new();
if (!allowHlsMultiExtMap)
{
isEndlist = true;
break;
}
}
}
// 评论行不解析
else if (line.StartsWith('#')) continue;
// 空白行不解析
else if (line.StartsWith("\r\n")) continue;
// 解析分片的地址
else if (expectSegment)
{
var segUrl = PreProcessUrl(ParserUtil.CombineURL(BaseUrl, line));
segment.Url = segUrl;
segments.Add(segment);
segment = new();
// YK的广告分段则清除此分片
// 需要注意，遇到广告说明程序对上文的#EXT-X-DISCONTINUITY做出的动作是不必要的，
// 其实上下文是同一种编码，需要恢复到原先的part上
if (segUrl.Contains("ccode=") && segUrl.Contains("/ad/") && segUrl.Contains("duration="))
{
segments.RemoveAt(segments.Count - 1);
segIndex--;
hasAd = true;
}
// YK广告(4K分辨率测试)
if (segUrl.Contains("ccode=0902") && segUrl.Contains("duration="))
{
segments.RemoveAt(segments.Count - 1);
segIndex--;
hasAd = true;
}
expectSegment = false;
}
}
// 直播的情况，无法遇到m3u8结束标记，需要手动将segments加入parts
if (!isEndlist)
{
mediaParts.Add(new MediaPart()
{
MediaSegments = segments
});
}
playlist.MediaParts = mediaParts;
playlist.IsLive = !isEndlist;
// 直播刷新间隔
if (playlist.IsLive)
{
// 由于播放器默认从最后3个分片开始播放 此处设置刷新间隔为TargetDuration的2倍
playlist.RefreshIntervalMs = (int)((playlist.TargetDuration ?? 5) * 2 * 1000);
}
return Task.FromResult(playlist);
}
private EncryptInfo ParseKey(string keyLine)
{
foreach (var p in ParserConfig.KeyProcessors)
{
if (p.CanProcess(ExtractorType, keyLine, M3u8Url, M3u8Content, ParserConfig))
{
// 匹配到对应处理器后不再继续
return p.Process(keyLine, M3u8Url, M3u8Content, ParserConfig);
}
}
throw new Exception(ResString.keyProcessorNotFound);
}
public async Task<List<StreamSpec>> ExtractStreamsAsync(string rawText)
{
this.M3u8Content = rawText;
this.PreProcessContent();
if (M3u8Content.Contains(HLSTags.ext_x_stream_inf))
{
Logger.Warn(ResString.masterM3u8Found);
var lists = await ParseMasterListAsync();
lists = lists.DistinctBy(p => p.Url).ToList();
return lists;
}
var playlist = await ParseListAsync();
return
[
new()
{
Url = ParserConfig.Url,
Playlist = playlist,
Extension = playlist.MediaInit != null ? "mp4" : "ts"
}
];
}
private async Task LoadM3u8FromUrlAsync(string url)
{
// Logger.Info(ResString.loadingUrl + url);
if (url.StartsWith("file:"))
{
var uri = new Uri(url);
this.M3u8Content = File.ReadAllText(uri.LocalPath);
}
else if (url.StartsWith("http"))
{
try
{
(this.M3u8Content, url) = await HTTPUtil.GetWebSourceAndNewUrlAsync(url, ParserConfig.Headers);
}
catch (HttpRequestException) when (url != ParserConfig.OriginalUrl)
{
// 当URL无法访问时，再请求原始URL
(this.M3u8Content, url) = await HTTPUtil.GetWebSourceAndNewUrlAsync(ParserConfig.OriginalUrl, ParserConfig.Headers);
}
}
this.M3u8Url = url;
this.SetBaseUrl();
this.PreProcessContent();
}
/// <summary>
/// 从Master链接中刷新各个流的URL
/// </summary>
/// <param name="lists"></param>
/// <returns></returns>
private async Task RefreshUrlFromMaster(List<StreamSpec> lists)
{
// 重新加载master m3u8, 刷新选中流的URL
await LoadM3u8FromUrlAsync(ParserConfig.Url);
var newStreams = await ParseMasterListAsync();
newStreams = newStreams.DistinctBy(p => p.Url).ToList();
foreach (var l in lists)
{
var match = newStreams.Where(n => n.ToShortString() == l.ToShortString()).ToList();
if (match.Count == 0) continue;
Logger.DebugMarkUp($"{l.Url} => {match.First().Url}");
l.Url = match.First().Url;
}
}
public async Task FetchPlayListAsync(List<StreamSpec> lists)
{
for (int i = 0; i < lists.Count; i++)
{
try
{
// 直接重新加载m3u8
await LoadM3u8FromUrlAsync(lists[i].Url!);
}
catch (HttpRequestException) when (MasterM3u8Flag)
{
Logger.WarnMarkUp("Can not load m3u8. Try refreshing url from master url...");
// 当前URL无法加载 尝试从Master链接中刷新URL
await RefreshUrlFromMaster(lists);
await LoadM3u8FromUrlAsync(lists[i].Url!);
}
var newPlaylist = await ParseListAsync();
if (lists[i].Playlist?.MediaInit != null)
lists[i].Playlist!.MediaParts = newPlaylist.MediaParts; // 不更新init
else
lists[i].Playlist = newPlaylist;
if (lists[i].MediaType == MediaType.SUBTITLES)
{
var a = lists[i].Playlist!.MediaParts.Any(p => p.MediaSegments.Any(m => m.Url.Contains(".ttml")));
var b = lists[i].Playlist!.MediaParts.Any(p => p.MediaSegments.Any(m => m.Url.Contains(".vtt") || m.Url.Contains(".webvtt")));
if (a) lists[i].Extension = "ttml";
if (b) lists[i].Extension = "vtt";
}
else
{
lists[i].Extension = lists[i].Playlist!.MediaInit != null ? "m4s" : "ts";
}
}
}
public async Task RefreshPlayListAsync(List<StreamSpec> streamSpecs)
{
await FetchPlayListAsync(streamSpecs);
}
}

21
src/N_m3u8DL-RE.Parser/Extractor/IExtractor.cs Normal file
View File

@ -0,0 +1,21 @@
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
namespace N_m3u8DL_RE.Parser.Extractor;
public interface IExtractor
{
ExtractorType ExtractorType { get; }
ParserConfig ParserConfig { get; set; }
Task<List<StreamSpec>> ExtractStreamsAsync(string rawText);
Task FetchPlayListAsync(List<StreamSpec> streamSpecs);
Task RefreshPlayListAsync(List<StreamSpec> streamSpecs);
string PreProcessUrl(string url);
void PreProcessContent();
}

52
src/N_m3u8DL-RE.Parser/Extractor/LiveTSExtractor.cs Normal file
View File

@ -0,0 +1,52 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Parser.Config;
namespace N_m3u8DL_RE.Parser.Extractor;
internal class LiveTSExtractor : IExtractor
{
public ExtractorType ExtractorType => ExtractorType.HTTP_LIVE;
public ParserConfig ParserConfig {get; set;}
public LiveTSExtractor(ParserConfig parserConfig)
{
this.ParserConfig = parserConfig;
}
public Task<List<StreamSpec>> ExtractStreamsAsync(string rawText)
{
return Task.FromResult(new List<StreamSpec>
{
new()
{
OriginalUrl = ParserConfig.OriginalUrl,
Url = ParserConfig.Url,
Playlist = new Playlist(),
GroupId = ResString.ReLiveTs
}
});
}
public Task FetchPlayListAsync(List<StreamSpec> streamSpecs)
{
throw new NotImplementedException();
}
public void PreProcessContent()
{
throw new NotImplementedException();
}
public string PreProcessUrl(string url)
{
throw new NotImplementedException();
}
public Task RefreshPlayListAsync(List<StreamSpec> streamSpecs)
{
throw new NotImplementedException();
}
}

387
src/N_m3u8DL-RE.Parser/Extractor/MSSExtractor.cs Normal file
View File

@ -0,0 +1,387 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Parser.Constants;
using N_m3u8DL_RE.Parser.Mp4;
using N_m3u8DL_RE.Parser.Util;
using System.Text.RegularExpressions;
using System.Xml.Linq;
namespace N_m3u8DL_RE.Parser.Extractor;
// Microsoft Smooth Streaming
// https://test.playready.microsoft.com/smoothstreaming/SSWSS720H264/SuperSpeedway_720.ism/manifest
// file:///C:/Users/nilaoda/Downloads/[MS-SSTR]-180316.pdf
internal partial class MSSExtractor : IExtractor
{
[GeneratedRegex("00000001\\d7([0-9a-fA-F]{6})")]
private static partial Regex VCodecsRegex();
////////////////////////////////////////
private static EncryptMethod DEFAULT_METHOD = EncryptMethod.CENC;
public ExtractorType ExtractorType => ExtractorType.MSS;
private string IsmUrl = string.Empty;
private string BaseUrl = string.Empty;
private string IsmContent = string.Empty;
public ParserConfig ParserConfig { get; set; }
public MSSExtractor(ParserConfig parserConfig)
{
this.ParserConfig = parserConfig;
SetInitUrl();
}
private void SetInitUrl()
{
this.IsmUrl = ParserConfig.Url ?? string.Empty;
this.BaseUrl = !string.IsNullOrEmpty(ParserConfig.BaseUrl) ? ParserConfig.BaseUrl : this.IsmUrl;
}
public Task<List<StreamSpec>> ExtractStreamsAsync(string rawText)
{
var streamList = new List<StreamSpec>();
this.IsmContent = rawText;
this.PreProcessContent();
var xmlDocument = XDocument.Parse(IsmContent);
// 选中第一个SmoothStreamingMedia节点
var ssmElement = xmlDocument.Elements().First(e => e.Name.LocalName == "SmoothStreamingMedia");
var timeScaleStr = ssmElement.Attribute("TimeScale")?.Value ?? "10000000";
var durationStr = ssmElement.Attribute("Duration")?.Value;
var timescale = Convert.ToInt32(timeScaleStr);
var isLiveStr = ssmElement.Attribute("IsLive")?.Value;
bool isLive = Convert.ToBoolean(isLiveStr ?? "FALSE");
var isProtection = false;
var protectionSystemId = "";
var protectionData = "";
// 加密检测
var protectElement = ssmElement.Elements().FirstOrDefault(e => e.Name.LocalName == "Protection");
if (protectElement != null)
{
var protectionHeader = protectElement.Element("ProtectionHeader");
if (protectionHeader != null)
{
isProtection = true;
protectionSystemId = protectionHeader.Attribute("SystemID")?.Value ?? "9A04F079-9840-4286-AB92-E65BE0885F95";
protectionData = HexUtil.BytesToHex(Convert.FromBase64String(protectionHeader.Value));
}
}
// 所有StreamIndex节点
var streamIndexElements = ssmElement.Elements().Where(e => e.Name.LocalName == "StreamIndex");
foreach (var streamIndex in streamIndexElements)
{
var type = streamIndex.Attribute("Type")?.Value; // "video" / "audio" / "text"
var name = streamIndex.Attribute("Name")?.Value;
var subType = streamIndex.Attribute("Subtype")?.Value; // text track
// 如果有则不从QualityLevel读取
// Bitrate = "{bitrate}" / "{Bitrate}"
// StartTimeSubstitution = "{start time}" / "{start_time}"
var urlPattern = streamIndex.Attribute("Url")?.Value;
var language = streamIndex.Attribute("Language")?.Value;
// 去除不规范的语言标签
if (language?.Length != 3) language = null;
// 所有c节点
var cElements = streamIndex.Elements().Where(e => e.Name.LocalName == "c");
// 所有QualityLevel节点
var qualityLevelElements = streamIndex.Elements().Where(e => e.Name.LocalName == "QualityLevel");
foreach (var qualityLevel in qualityLevelElements)
{
urlPattern = (qualityLevel.Attribute("Url")?.Value ?? urlPattern)!
.Replace(MSSTags.Bitrate_BK, MSSTags.Bitrate).Replace(MSSTags.StartTime_BK, MSSTags.StartTime);
var fourCC = qualityLevel.Attribute("FourCC")!.Value.ToUpper();
var samplingRateStr = qualityLevel.Attribute("SamplingRate")?.Value;
var bitsPerSampleStr = qualityLevel.Attribute("BitsPerSample")?.Value;
var nalUnitLengthFieldStr = qualityLevel.Attribute("NALUnitLengthField")?.Value;
var indexStr = qualityLevel.Attribute("Index")?.Value;
var codecPrivateData = qualityLevel.Attribute("CodecPrivateData")?.Value ?? "";
var audioTag = qualityLevel.Attribute("AudioTag")?.Value;
var bitrate = Convert.ToInt32(qualityLevel.Attribute("Bitrate")?.Value ?? "0");
var width = Convert.ToInt32(qualityLevel.Attribute("MaxWidth")?.Value ?? "0");
var height = Convert.ToInt32(qualityLevel.Attribute("MaxHeight")?.Value ?? "0");
var channels = qualityLevel.Attribute("Channels")?.Value;
StreamSpec streamSpec = new();
streamSpec.PublishTime = DateTime.Now; // 发布时间默认现在
streamSpec.Extension = "m4s";
streamSpec.OriginalUrl = ParserConfig.OriginalUrl;
streamSpec.PeriodId = indexStr;
streamSpec.Playlist = new Playlist();
streamSpec.Playlist.IsLive = isLive;
streamSpec.Playlist.MediaParts.Add(new MediaPart());
streamSpec.GroupId = name ?? indexStr;
streamSpec.Bandwidth = bitrate;
streamSpec.Codecs = ParseCodecs(fourCC, codecPrivateData);
streamSpec.Language = language;
streamSpec.Resolution = width == 0 ? null : $"{width}x{height}";
streamSpec.Url = IsmUrl;
streamSpec.Channels = channels;
streamSpec.MediaType = type switch
{
"text" => MediaType.SUBTITLES,
"audio" => MediaType.AUDIO,
_ => null
};
streamSpec.Playlist.MediaInit = new MediaSegment();
if (!string.IsNullOrEmpty(codecPrivateData))
{
streamSpec.Playlist.MediaInit.Index = -1; // 便于排序
streamSpec.Playlist.MediaInit.Url = $"hex://{codecPrivateData}";
}
var currentTime = 0L;
var segIndex = 0;
var varDic = new Dictionary<string, object?>();
varDic[MSSTags.Bitrate] = bitrate;
foreach (var c in cElements)
{
// 每个C元素包含三个属性:@t(start time)\@r(repeat count)\@d(duration)
var _startTimeStr = c.Attribute("t")?.Value;
var _durationStr = c.Attribute("d")?.Value;
var _repeatCountStr = c.Attribute("r")?.Value;
if (_startTimeStr != null) currentTime = Convert.ToInt64(_startTimeStr);
var _duration = Convert.ToInt64(_durationStr);
var _repeatCount = Convert.ToInt64(_repeatCountStr);
if (_repeatCount > 0)
{
// This value is one-based. (A value of 2 means two fragments in the contiguous series).
_repeatCount -= 1;
}
varDic[MSSTags.StartTime] = currentTime;
var oriUrl = ParserUtil.CombineURL(this.BaseUrl, urlPattern!);
var mediaUrl = ParserUtil.ReplaceVars(oriUrl, varDic);
MediaSegment mediaSegment = new();
mediaSegment.Url = mediaUrl;
if (oriUrl.Contains(MSSTags.StartTime))
mediaSegment.NameFromVar = currentTime.ToString();
mediaSegment.Duration = _duration / (double)timescale;
mediaSegment.Index = segIndex++;
streamSpec.Playlist.MediaParts[0].MediaSegments.Add(mediaSegment);
if (_repeatCount < 0)
{
// 负数表示一直重复 直到period结束 注意减掉已经加入的1个片段
_repeatCount = (long)Math.Ceiling(Convert.ToInt64(durationStr) / (double)_duration) - 1;
}
for (long i = 0; i < _repeatCount; i++)
{
currentTime += _duration;
MediaSegment _mediaSegment = new();
varDic[MSSTags.StartTime] = currentTime;
var _oriUrl = ParserUtil.CombineURL(this.BaseUrl, urlPattern!);
var _mediaUrl = ParserUtil.ReplaceVars(_oriUrl, varDic);
_mediaSegment.Url = _mediaUrl;
_mediaSegment.Index = segIndex++;
_mediaSegment.Duration = _duration / (double)timescale;
if (_oriUrl.Contains(MSSTags.StartTime))
_mediaSegment.NameFromVar = currentTime.ToString();
streamSpec.Playlist.MediaParts[0].MediaSegments.Add(_mediaSegment);
}
currentTime += _duration;
}
// 生成MOOV数据
if (MSSMoovProcessor.CanHandle(fourCC!))
{
streamSpec.MSSData = new MSSData()
{
FourCC = fourCC!,
CodecPrivateData = codecPrivateData,
Type = type!,
Timesacle = Convert.ToInt32(timeScaleStr),
Duration = Convert.ToInt64(durationStr),
SamplingRate = Convert.ToInt32(samplingRateStr ?? "48000"),
Channels = Convert.ToInt32(channels ?? "2"),
BitsPerSample = Convert.ToInt32(bitsPerSampleStr ?? "16"),
NalUnitLengthField = Convert.ToInt32(nalUnitLengthFieldStr ?? "4"),
IsProtection = isProtection,
ProtectionData = protectionData,
ProtectionSystemID = protectionSystemId,
};
var processor = new MSSMoovProcessor(streamSpec);
var header = processor.GenHeader(); // trackId可能不正确
streamSpec.Playlist!.MediaInit!.Url = $"base64://{Convert.ToBase64String(header)}";
// 为音视频写入加密信息
if (isProtection && type != "text")
{
if (streamSpec.Playlist.MediaInit != null)
{
streamSpec.Playlist.MediaInit.EncryptInfo.Method = DEFAULT_METHOD;
}
foreach (var item in streamSpec.Playlist.MediaParts[0].MediaSegments)
{
item.EncryptInfo.Method = DEFAULT_METHOD;
}
}
streamList.Add(streamSpec);
}
else
{
Logger.WarnMarkUp($"[green]{fourCC}[/] not supported! Skiped.");
}
}
}
// 为视频设置默认轨道
var aL = streamList.Where(s => s.MediaType == MediaType.AUDIO).ToList();
var sL = streamList.Where(s => s.MediaType == MediaType.SUBTITLES).ToList();
foreach (var item in streamList.Where(item => !string.IsNullOrEmpty(item.Resolution)))
{
if (aL.Count != 0)
{
item.AudioId = aL.First().GroupId;
}
if (sL.Count != 0)
{
item.SubtitleId = sL.First().GroupId;
}
}
return Task.FromResult(streamList);
}
/// <summary>
/// 解析编码
/// </summary>
/// <param name="fourCC"></param>
/// <returns></returns>
private static string? ParseCodecs(string fourCC, string? privateData)
{
if (fourCC == "TTML") return "stpp";
if (string.IsNullOrEmpty(privateData)) return null;
return fourCC switch
{
// AVC视频
"H264" or "X264" or "DAVC" or "AVC1" => ParseAVCCodecs(privateData),
// AAC音频
"AAC" or "AACL" or "AACH" or "AACP" => ParseAACCodecs(fourCC, privateData),
// 默认返回fourCC本身
_ => fourCC.ToLower()
};
}
private static string ParseAVCCodecs(string privateData)
{
var result = VCodecsRegex().Match(privateData).Groups[1].Value;
return string.IsNullOrEmpty(result) ? "avc1.4D401E" : $"avc1.{result}";
}
private static string ParseAACCodecs(string fourCC, string privateData)
{
var mpProfile = 2;
if (fourCC == "AACH")
{
mpProfile = 5; // High Efficiency AAC Profile
}
else if (!string.IsNullOrEmpty(privateData))
{
mpProfile = (Convert.ToByte(privateData[..2], 16) & 0xF8) >> 3;
}
return $"mp4a.40.{mpProfile}";
}
public async Task FetchPlayListAsync(List<StreamSpec> streamSpecs)
{
// 这里才调用URL预处理器，节省开销
await ProcessUrlAsync(streamSpecs);
}
private Task ProcessUrlAsync(List<StreamSpec> streamSpecs)
{
foreach (var streamSpec in streamSpecs)
{
var playlist = streamSpec.Playlist;
if (playlist == null) continue;
if (playlist.MediaInit != null)
{
playlist.MediaInit!.Url = PreProcessUrl(playlist.MediaInit!.Url);
}
for (var ii = 0; ii < playlist!.MediaParts.Count; ii++)
{
var part = playlist.MediaParts[ii];
foreach (var segment in part.MediaSegments)
{
segment.Url = PreProcessUrl(segment.Url);
}
}
}
return Task.CompletedTask;
}
public string PreProcessUrl(string url)
{
foreach (var p in ParserConfig.UrlProcessors)
{
if (p.CanProcess(ExtractorType, url, ParserConfig))
{
url = p.Process(url, ParserConfig);
}
}
return url;
}
public void PreProcessContent()
{
foreach (var p in ParserConfig.ContentProcessors)
{
if (p.CanProcess(ExtractorType, IsmContent, ParserConfig))
{
IsmContent = p.Process(IsmContent, ParserConfig);
}
}
}
public async Task RefreshPlayListAsync(List<StreamSpec> streamSpecs)
{
if (streamSpecs.Count == 0) return;
var (rawText, url) = ("", ParserConfig.Url);
try
{
(rawText, url) = await HTTPUtil.GetWebSourceAndNewUrlAsync(ParserConfig.Url, ParserConfig.Headers);
}
catch (HttpRequestException) when (ParserConfig.Url != ParserConfig.OriginalUrl)
{
// 当URL无法访问时，再请求原始URL
(rawText, url) = await HTTPUtil.GetWebSourceAndNewUrlAsync(ParserConfig.OriginalUrl, ParserConfig.Headers);
}
ParserConfig.Url = url;
SetInitUrl();
var newStreams = await ExtractStreamsAsync(rawText);
foreach (var streamSpec in streamSpecs)
{
// 有的网站每次请求MPD返回的码率不一致，导致ToShortString()无法匹配 无法更新playlist
// 故增加通过init url来匹配 (如果有的话)
var match = newStreams.Where(n => n.ToShortString() == streamSpec.ToShortString());
if (!match.Any())
match = newStreams.Where(n => n.Playlist?.MediaInit?.Url == streamSpec.Playlist?.MediaInit?.Url);
if (match.Any())
streamSpec.Playlist!.MediaParts = match.First().Playlist!.MediaParts; // 不更新init
}
// 这里才调用URL预处理器，节省开销
await ProcessUrlAsync(streamSpecs);
}
}

62
src/N_m3u8DL-RE.Parser/Mp4/BinaryReader2.cs Normal file
View File

@ -0,0 +1,62 @@
namespace Mp4SubtitleParser;
// make BinaryReader in Big Endian
class BinaryReader2 : BinaryReader
{
public BinaryReader2(System.IO.Stream stream) : base(stream) { }
public bool HasMoreData()
{
return BaseStream.Position < BaseStream.Length;
}
public long GetLength()
{
return BaseStream.Length;
}
public long GetPosition()
{
return BaseStream.Position;
}
public override int ReadInt32()
{
var data = base.ReadBytes(4);
if (BitConverter.IsLittleEndian)
Array.Reverse(data);
return BitConverter.ToInt32(data, 0);
}
public override short ReadInt16()
{
var data = base.ReadBytes(2);
if (BitConverter.IsLittleEndian)
Array.Reverse(data);
return BitConverter.ToInt16(data, 0);
}
public override long ReadInt64()
{
var data = base.ReadBytes(8);
if (BitConverter.IsLittleEndian)
Array.Reverse(data);
return BitConverter.ToInt64(data, 0);
}
public override uint ReadUInt32()
{
var data = base.ReadBytes(4);
if (BitConverter.IsLittleEndian)
Array.Reverse(data);
return BitConverter.ToUInt32(data, 0);
}
public override ulong ReadUInt64()
{
var data = base.ReadBytes(8);
if (BitConverter.IsLittleEndian)
Array.Reverse(data);
return BitConverter.ToUInt64(data, 0);
}
}

83
src/N_m3u8DL-RE.Parser/Mp4/BinaryWriter2.cs Normal file
View File

@ -0,0 +1,83 @@
using System.Text;
namespace Mp4SubtitleParser;
// make BinaryWriter in Big Endian
class BinaryWriter2 : BinaryWriter
{
private static bool IsLittleEndian = BitConverter.IsLittleEndian;
public BinaryWriter2(System.IO.Stream stream) : base(stream) { }
public void WriteUInt(decimal n, int offset = 0)
{
var arr = BitConverter.GetBytes((uint)n);
if (IsLittleEndian)
Array.Reverse(arr);
if (offset != 0)
arr = arr[offset..];
BaseStream.Write(arr);
}
public override void Write(string text)
{
BaseStream.Write(Encoding.ASCII.GetBytes(text));
}
public void WriteInt(decimal n, int offset = 0)
{
var arr = BitConverter.GetBytes((int)n);
if (IsLittleEndian)
Array.Reverse(arr);
if (offset != 0)
arr = arr[offset..];
BaseStream.Write(arr);
}
public void WriteULong(decimal n, int offset = 0)
{
var arr = BitConverter.GetBytes((ulong)n);
if (IsLittleEndian)
Array.Reverse(arr);
if (offset != 0)
arr = arr[offset..];
BaseStream.Write(arr);
}
public void WriteUShort(decimal n, int padding = 0)
{
var arr = BitConverter.GetBytes((ushort)n);
if (IsLittleEndian)
Array.Reverse(arr);
while (padding > 0)
{
arr = arr.Concat(new byte[] { 0x00 }).ToArray();
padding--;
}
BaseStream.Write(arr);
}
public void WriteShort(decimal n, int padding = 0)
{
var arr = BitConverter.GetBytes((short)n);
if (IsLittleEndian)
Array.Reverse(arr);
while (padding > 0)
{
arr = arr.Concat(new byte[] { 0x00 }).ToArray();
padding--;
}
BaseStream.Write(arr);
}
public void WriteByte(byte n, int padding = 0)
{
var arr = new byte[] { n };
while (padding > 0)
{
arr = arr.Concat(new byte[] { 0x00 }).ToArray();
padding--;
}
BaseStream.Write(arr);
}
}

91
src/N_m3u8DL-RE.Parser/Mp4/MP4InitUtil.cs Normal file
View File

@ -0,0 +1,91 @@
using N_m3u8DL_RE.Common.Util;
namespace Mp4SubtitleParser
{
public class ParsedMP4Info
{
public string? PSSH;
public string? KID;
public string? Scheme;
public bool isMultiDRM;
}
public static class MP4InitUtil
{
private static readonly byte[] SYSTEM_ID_WIDEVINE = [0xED, 0xEF, 0x8B, 0xA9, 0x79, 0xD6, 0x4A, 0xCE, 0xA3, 0xC8, 0x27, 0xDC, 0xD5, 0x1D, 0x21, 0xED];
private static readonly byte[] SYSTEM_ID_PLAYREADY = [0x9A, 0x04, 0xF0, 0x79, 0x98, 0x40, 0x42, 0x86, 0xAB, 0x92, 0xE6, 0x5B, 0xE0, 0x88, 0x5F, 0x95];
public static ParsedMP4Info ReadInit(byte[] data)
{
var info = new ParsedMP4Info();
// parse init
new MP4Parser()
.Box("moov", MP4Parser.Children)
.Box("trak", MP4Parser.Children)
.Box("mdia", MP4Parser.Children)
.Box("minf", MP4Parser.Children)
.Box("stbl", MP4Parser.Children)
.FullBox("stsd", MP4Parser.SampleDescription)
.FullBox("pssh", box =>
{
if (box.Version is not (0 or 1))
throw new Exception("PSSH version can only be 0 or 1");
var systemId = box.Reader.ReadBytes(16);
if (!SYSTEM_ID_WIDEVINE.SequenceEqual(systemId)) return;
var dataSize = box.Reader.ReadUInt32();
var psshData = box.Reader.ReadBytes((int)dataSize);
info.PSSH = Convert.ToBase64String(psshData);
if (info.KID != "00000000000000000000000000000000") return;
info.KID = HexUtil.BytesToHex(psshData[2..18]).ToLower();
info.isMultiDRM = true;
})
.FullBox("encv", MP4Parser.AllData(data => ReadBox(data, info)))
.FullBox("enca", MP4Parser.AllData(data => ReadBox(data, info)))
.FullBox("enct", MP4Parser.AllData(data => ReadBox(data, info)))
.FullBox("encs", MP4Parser.AllData(data => ReadBox(data, info)))
.Parse(data, stopOnPartial: true);
return info;
}
private static void ReadBox(byte[] data, ParsedMP4Info info)
{
// find schm
byte[] schmBytes = [0x73, 0x63, 0x68, 0x6d];
var schmIndex = 0;
for (var i = 0; i < data.Length - 4; i++)
{
if (new[] { data[i], data[i + 1], data[i + 2], data[i + 3] }.SequenceEqual(schmBytes))
{
schmIndex = i;
break;
}
}
if (schmIndex + 8 < data.Length)
{
info.Scheme = System.Text.Encoding.UTF8.GetString(data[schmIndex..][8..12]);
}
// if (info.Scheme != "cenc") return;
// find KID
byte[] tencBytes = [0x74, 0x65, 0x6E, 0x63];
var tencIndex = -1;
for (int i = 0; i < data.Length - 4; i++)
{
if (new[] { data[i], data[i + 1], data[i + 2], data[i + 3] }.SequenceEqual(tencBytes))
{
tencIndex = i;
break;
}
}
if (tencIndex != -1 && tencIndex + 12 < data.Length)
{
info.KID = HexUtil.BytesToHex(data[tencIndex..][12..28]).ToLower();
}
}
}
}

344
src/N_m3u8DL-RE.Parser/Mp4/MP4Parser.cs Normal file
View File

@ -0,0 +1,344 @@
using System.Text;
/**
* Translated from shaka-player project
* https://github.com/nilaoda/Mp4SubtitleParser
* https://github.com/shaka-project/shaka-player
*/
namespace Mp4SubtitleParser
{
class ParsedBox
{
public required MP4Parser Parser { get; set; }
public bool PartialOkay { get; set; }
public long Start { get; set; }
public uint Version { get; set; } = 1000;
public uint Flags { get; set; } = 1000;
public required BinaryReader2 Reader { get; set; }
public bool Has64BitSize { get; set; }
}
class TFHD
{
public uint TrackId { get; set; }
public uint DefaultSampleDuration { get; set; }
public uint DefaultSampleSize { get; set; }
}
class TRUN
{
public uint SampleCount { get; set; }
public List<Sample> SampleData { get; set; } = [];
}
class Sample
{
public uint SampleDuration { get; set; }
public uint SampleSize { get; set; }
public uint SampleCompositionTimeOffset { get; set; }
}
enum BoxType
{
BASIC_BOX = 0,
FULL_BOX = 1
};
class MP4Parser
{
public bool Done { get; set; } = false;
public Dictionary<long, int> Headers { get; set; } = new Dictionary<long, int>();
public Dictionary<long, BoxHandler> BoxDefinitions { get; set; } = new Dictionary<long, BoxHandler>();
public delegate void BoxHandler(ParsedBox box);
public delegate void DataHandler(byte[] data);
public static BoxHandler AllData(DataHandler handler)
{
return box =>
{
var all = box.Reader.GetLength() - box.Reader.GetPosition();
handler(box.Reader.ReadBytes((int)all));
};
}
public static void Children(ParsedBox box)
{
var headerSize = HeaderSize(box);
while (box.Reader.HasMoreData() && !box.Parser.Done)
{
box.Parser.ParseNext(box.Start + headerSize, box.Reader, box.PartialOkay);
}
}
public static void SampleDescription(ParsedBox box)
{
var headerSize = HeaderSize(box);
var count = box.Reader.ReadUInt32();
for (int i = 0; i < count; i++)
{
box.Parser.ParseNext(box.Start + headerSize, box.Reader, box.PartialOkay);
if (box.Parser.Done)
{
break;
}
}
}
public void Parse(byte[] data, bool partialOkay = false, bool stopOnPartial = false)
{
var reader = new BinaryReader2(new MemoryStream(data));
this.Done = false;
while (reader.HasMoreData() && !this.Done)
{
this.ParseNext(0, reader, partialOkay, stopOnPartial);
}
}
private void ParseNext(long absStart, BinaryReader2 reader, bool partialOkay, bool stopOnPartial = false)
{
var start = reader.GetPosition();
// size(4 bytes) + type(4 bytes) = 8 bytes
if (stopOnPartial && start + 8 > reader.GetLength())
{
this.Done = true;
return;
}
long size = reader.ReadUInt32();
long type = reader.ReadUInt32();
var name = TypeToString(type);
var has64BitSize = false;
// Console.WriteLine($"Parsing MP4 box: {name}");
switch (size)
{
case 0:
size = reader.GetLength() - start;
break;
case 1:
if (stopOnPartial && reader.GetPosition() + 8 > reader.GetLength())
{
this.Done = true;
return;
}
size = (long)reader.ReadUInt64();
has64BitSize = true;
break;
}
this.BoxDefinitions.TryGetValue(type, out BoxHandler? boxDefinition);
if (boxDefinition != null)
{
uint version = 1000;
uint flags = 1000;
if (this.Headers[type] == (int)BoxType.FULL_BOX)
{
if (stopOnPartial && reader.GetPosition() + 4 > reader.GetLength())
{
this.Done = true;
return;
}
var versionAndFlags = reader.ReadUInt32();
version = versionAndFlags >> 24;
flags = versionAndFlags & 0xFFFFFF;
}
var end = start + size;
if (partialOkay && end > reader.GetLength())
{
// For partial reads, truncate the payload if we must.
end = reader.GetLength();
}
if (stopOnPartial && end > reader.GetLength())
{
this.Done = true;
return;
}
int payloadSize = (int)(end - reader.GetPosition());
var payload = (payloadSize > 0) ? reader.ReadBytes(payloadSize) : [];
var box = new ParsedBox()
{
Parser = this,
PartialOkay = partialOkay || false,
Version = version,
Flags = flags,
Reader = new BinaryReader2(new MemoryStream(payload)),
Start = start + absStart,
Has64BitSize = has64BitSize,
};
boxDefinition(box);
}
else
{
// Move the read head to be at the end of the box.
// If the box is longer than the remaining parts of the file, e.g. the
// mp4 is improperly formatted, or this was a partial range request that
// ended in the middle of a box, just skip to the end.
var skipLength = Math.Min(
start + size - reader.GetPosition(),
reader.GetLength() - reader.GetPosition());
reader.ReadBytes((int)skipLength);
}
}
private static int HeaderSize(ParsedBox box)
{
return /* basic header */ 8
+ /* additional 64-bit size field */ (box.Has64BitSize ? 8 : 0)
+ /* version and flags for a "full" box */ (box.Flags != 0 ? 4 : 0);
}
public static string TypeToString(long type)
{
return Encoding.UTF8.GetString(new byte[]
{
(byte)((type >> 24) & 0xff),
(byte)((type >> 16) & 0xff),
(byte)((type >> 8) & 0xff),
(byte)(type & 0xff)
});
}
private static int TypeFromString(string name)
{
if (name.Length != 4)
throw new Exception("Mp4 box names must be 4 characters long");
var code = 0;
foreach (var chr in name) {
code = (code << 8) | chr;
}
return code;
}
public MP4Parser Box(string type, BoxHandler handler)
{
var typeCode = TypeFromString(type);
this.Headers[typeCode] = (int)BoxType.BASIC_BOX;
this.BoxDefinitions[typeCode] = handler;
return this;
}
public MP4Parser FullBox(string type, BoxHandler handler)
{
var typeCode = TypeFromString(type);
this.Headers[typeCode] = (int)BoxType.FULL_BOX;
this.BoxDefinitions[typeCode] = handler;
return this;
}
public static uint ParseMDHD(BinaryReader2 reader, uint version)
{
if (version == 1)
{
reader.ReadBytes(8); // Skip "creation_time"
reader.ReadBytes(8); // Skip "modification_time"
}
else
{
reader.ReadBytes(4); // Skip "creation_time"
reader.ReadBytes(4); // Skip "modification_time"
}
return reader.ReadUInt32();
}
public static ulong ParseTFDT(BinaryReader2 reader, uint version)
{
return version == 1 ? reader.ReadUInt64() : reader.ReadUInt32();
}
public static TFHD ParseTFHD(BinaryReader2 reader, uint flags)
{
var trackId = reader.ReadUInt32();
uint defaultSampleDuration = 0;
uint defaultSampleSize = 0;
// Skip "base_data_offset" if present.
if ((flags & 0x000001) != 0)
{
reader.ReadBytes(8);
}
// Skip "sample_description_index" if present.
if ((flags & 0x000002) != 0)
{
reader.ReadBytes(4);
}
// Read "default_sample_duration" if present.
if ((flags & 0x000008) != 0)
{
defaultSampleDuration = reader.ReadUInt32();
}
// Read "default_sample_size" if present.
if ((flags & 0x000010) != 0)
{
defaultSampleSize = reader.ReadUInt32();
}
return new TFHD() { TrackId = trackId, DefaultSampleDuration = defaultSampleDuration, DefaultSampleSize = defaultSampleSize };
}
public static TRUN ParseTRUN(BinaryReader2 reader, uint version, uint flags)
{
var trun = new TRUN();
trun.SampleCount = reader.ReadUInt32();
// Skip "data_offset" if present.
if ((flags & 0x000001) != 0)
{
reader.ReadBytes(4);
}
// Skip "first_sample_flags" if present.
if ((flags & 0x000004) != 0)
{
reader.ReadBytes(4);
}
for (int i = 0; i < trun.SampleCount; i++)
{
var sample = new Sample();
// Read "sample duration" if present.
if ((flags & 0x000100) != 0)
{
sample.SampleDuration = reader.ReadUInt32();
}
// Read "sample_size" if present.
if ((flags & 0x000200) != 0)
{
sample.SampleSize = reader.ReadUInt32();
}
// Skip "sample_flags" if present.
if ((flags & 0x000400) != 0)
{
reader.ReadBytes(4);
}
// Read "sample_time_offset" if present.
if ((flags & 0x000800) != 0)
{
sample.SampleCompositionTimeOffset = version == 0 ?
reader.ReadUInt32() :
(uint)reader.ReadInt32();
}
trun.SampleData.Add(sample);
}
return trun;
}
}
}

374
src/N_m3u8DL-RE.Parser/Mp4/MP4TtmlUtil.cs Normal file
View File

@ -0,0 +1,374 @@
using N_m3u8DL_RE.Common.Entity;
using System.Text;
using System.Text.RegularExpressions;
using System.Xml;
namespace Mp4SubtitleParser;
class SubEntity
{
public required string Begin { get; set; }
public required string End { get; set; }
public required string Region { get; set; }
public List<XmlElement> Contents { get; set; } = [];
public List<string> ContentStrings { get; set; } = [];
public override bool Equals(object? obj)
{
return obj is SubEntity entity &&
Begin == entity.Begin &&
End == entity.End &&
Region == entity.Region &&
ContentStrings.SequenceEqual(entity.ContentStrings);
}
public override int GetHashCode()
{
return HashCode.Combine(Begin, End, Region, ContentStrings);
}
}
public static partial class MP4TtmlUtil
{
[GeneratedRegex(" \\w+:\\w+=\\\"[^\\\"]*\\\"")]
private static partial Regex AttrRegex();
[GeneratedRegex("<p.*?>(.+?)<\\/p>")]
private static partial Regex LabelFixRegex();
[GeneratedRegex(@"\<tt[\s\S]*?\<\/tt\>")]
private static partial Regex MultiElementsFixRegex();
[GeneratedRegex("\\<smpte:image.*xml:id=\\\"(.*?)\\\".*\\>([\\s\\S]*?)<\\/smpte:image>")]
private static partial Regex ImageRegex();
public static bool CheckInit(byte[] data)
{
bool sawSTPP = false;
// parse init
new MP4Parser()
.Box("moov", MP4Parser.Children)
.Box("trak", MP4Parser.Children)
.Box("mdia", MP4Parser.Children)
.Box("minf", MP4Parser.Children)
.Box("stbl", MP4Parser.Children)
.FullBox("stsd", MP4Parser.SampleDescription)
.Box("stpp", box => {
sawSTPP = true;
})
.Parse(data);
return sawSTPP;
}
private static string ShiftTime(string xmlSrc, long segTimeMs, int index)
{
string Add(string xmlTime)
{
var dt = DateTime.ParseExact(xmlTime, "HH:mm:ss.fff", System.Globalization.CultureInfo.InvariantCulture);
var ts = TimeSpan.FromMilliseconds(dt.TimeOfDay.TotalMilliseconds + segTimeMs * index);
return $"{ts.Hours:00}:{ts.Minutes:00}:{ts.Seconds:00}.{ts.Milliseconds:000}";
}
if (!xmlSrc.Contains("<tt") || !xmlSrc.Contains("<head>")) return xmlSrc;
var xmlDoc = new XmlDocument();
XmlNamespaceManager? nsMgr = null;
xmlDoc.LoadXml(xmlSrc);
var ttNode = xmlDoc.LastChild;
if (nsMgr == null)
{
var ns = ((XmlElement)ttNode!).GetAttribute("xmlns");
nsMgr = new XmlNamespaceManager(xmlDoc.NameTable);
nsMgr.AddNamespace("ns", ns);
}
var bodyNode = ttNode!.SelectSingleNode("ns:body", nsMgr);
if (bodyNode == null)
return xmlSrc;
var _div = bodyNode.SelectSingleNode("ns:div", nsMgr);
// Parse <p> label
foreach (XmlElement _p in _div!.SelectNodes("ns:p", nsMgr)!)
{
var _begin = _p.GetAttribute("begin");
var _end = _p.GetAttribute("end");
// Handle namespace
foreach (XmlAttribute attr in _p.Attributes)
{
if (attr.LocalName == "begin") _begin = attr.Value;
else if (attr.LocalName == "end") _end = attr.Value;
}
_p.SetAttribute("begin", Add(_begin));
_p.SetAttribute("end", Add(_end));
// Console.WriteLine($"{_begin} {_p.GetAttribute("begin")}");
// Console.WriteLine($"{_end} {_p.GetAttribute("begin")}");
}
return xmlDoc.OuterXml;
}
private static string GetTextFromElement(XmlElement node)
{
var sb = new StringBuilder();
foreach (XmlNode item in node.ChildNodes)
{
if (item.NodeType == XmlNodeType.Text)
{
sb.Append(item.InnerText.Trim());
}
else if(item is { NodeType: XmlNodeType.Element, Name: "br" })
{
sb.AppendLine();
}
}
return sb.ToString();
}
private static List<string> SplitMultipleRootElements(string xml)
{
return !MultiElementsFixRegex().IsMatch(xml) ? [] : MultiElementsFixRegex().Matches(xml).Select(m => m.Value).ToList();
}
public static WebVttSub ExtractFromMp4(string item, long segTimeMs, long baseTimestamp = 0L)
{
return ExtractFromMp4s([item], segTimeMs, baseTimestamp);
}
private static WebVttSub ExtractFromMp4s(IEnumerable<string> items, long segTimeMs, long baseTimestamp = 0L)
{
// read ttmls
List<string> xmls = [];
int segIndex = 0;
foreach (var item in items)
{
var dataSeg = File.ReadAllBytes(item);
var sawMDAT = false;
// parse media
new MP4Parser()
.Box("mdat", MP4Parser.AllData(data =>
{
sawMDAT = true;
// Join this to any previous payload, in case the mp4 has multiple
// mdats.
if (segTimeMs != 0)
{
var datas = SplitMultipleRootElements(Encoding.UTF8.GetString(data));
foreach (var item in datas)
{
xmls.Add(ShiftTime(item, segTimeMs, segIndex));
}
}
else
{
var datas = SplitMultipleRootElements(Encoding.UTF8.GetString(data));
xmls.AddRange(datas);
}
}))
.Parse(dataSeg,/* partialOkay= */ false);
segIndex++;
}
return ExtractSub(xmls, baseTimestamp);
}
public static WebVttSub ExtractFromTTML(string item, long segTimeMs, long baseTimestamp = 0L)
{
return ExtractFromTTMLs([item], segTimeMs, baseTimestamp);
}
public static WebVttSub ExtractFromTTMLs(IEnumerable<string> items, long segTimeMs, long baseTimestamp = 0L)
{
// read ttmls
List<string> xmls = [];
int segIndex = 0;
foreach (var item in items)
{
var xml = File.ReadAllText(item);
xmls.Add(segTimeMs != 0 ? ShiftTime(xml, segTimeMs, segIndex) : xml);
segIndex++;
}
return ExtractSub(xmls, baseTimestamp);
}
private static WebVttSub ExtractSub(List<string> xmls, long baseTimestamp)
{
// parsing
var xmlDoc = new XmlDocument();
var finalSubs = new List<SubEntity>();
XmlNode? headNode = null;
XmlNamespaceManager? nsMgr = null;
var regex = LabelFixRegex();
var attrRegex = AttrRegex();
foreach (var item in xmls)
{
var xmlContent = item;
if (!xmlContent.Contains("<tt")) continue;
// fix non-standard xml
var xmlContentFix = xmlContent;
if (regex.IsMatch(xmlContent))
{
foreach (Match m in regex.Matches(xmlContentFix))
{
try
{
var inner = m.Groups[1].Value;
if (attrRegex.IsMatch(inner))
{
inner = attrRegex.Replace(inner, "");
}
new XmlDocument().LoadXml($"<p>{inner}</p>");
}
catch (Exception)
{
xmlContentFix = xmlContentFix.Replace(m.Groups[1].Value, System.Web.HttpUtility.HtmlEncode(m.Groups[1].Value));
}
}
}
xmlDoc.LoadXml(xmlContentFix);
var ttNode = xmlDoc.LastChild;
if (nsMgr == null)
{
var ns = ((XmlElement)ttNode!).GetAttribute("xmlns");
nsMgr = new XmlNamespaceManager(xmlDoc.NameTable);
nsMgr.AddNamespace("ns", ns);
}
if (headNode == null)
headNode = ttNode!.SelectSingleNode("ns:head", nsMgr);
var bodyNode = ttNode!.SelectSingleNode("ns:body", nsMgr);
if (bodyNode == null)
continue;
var _div = bodyNode.SelectSingleNode("ns:div", nsMgr);
if (_div == null)
continue;
// PNG Subs
var imageDic = new Dictionary<string, string>(); // id, Base64
if (ImageRegex().IsMatch(xmlDoc.InnerXml))
{
foreach (Match img in ImageRegex().Matches(xmlDoc.InnerXml))
{
imageDic.Add(img.Groups[1].Value.Trim(), img.Groups[2].Value.Trim());
}
}
// convert <div> to <p>
if (_div!.SelectNodes("ns:p", nsMgr) == null || _div!.SelectNodes("ns:p", nsMgr)!.Count == 0)
{
foreach (XmlElement _tDiv in bodyNode.SelectNodes("ns:div", nsMgr)!)
{
var _p = xmlDoc.CreateDocumentFragment();
_p.InnerXml = _tDiv.OuterXml.Replace("<div ", "<p ").Replace("</div>", "</p>");
_div.AppendChild(_p);
}
}
// Parse <p> label
foreach (XmlElement _p in _div!.SelectNodes("ns:p", nsMgr)!)
{
var _begin = _p.GetAttribute("begin");
var _end = _p.GetAttribute("end");
var _region = _p.GetAttribute("region");
var _bgImg = _p.GetAttribute("smpte:backgroundImage");
// Handle namespace
foreach (XmlAttribute attr in _p.Attributes)
{
if (attr.LocalName == "begin") _begin = attr.Value;
else if (attr.LocalName == "end") _end = attr.Value;
else if (attr.LocalName == "region") _region = attr.Value;
}
var sub = new SubEntity
{
Begin = _begin,
End = _end,
Region = _region
};
if (string.IsNullOrEmpty(_bgImg))
{
var _spans = _p.ChildNodes;
// Collect <span>
foreach (XmlNode _node in _spans)
{
if (_node.NodeType == XmlNodeType.Element)
{
var _span = (XmlElement)_node;
if (string.IsNullOrEmpty(_span.InnerText))
continue;
sub.Contents.Add(_span);
sub.ContentStrings.Add(_span.OuterXml);
}
else if (_node.NodeType == XmlNodeType.Text)
{
var _span = new XmlDocument().CreateElement("span");
_span.InnerText = _node.Value!;
sub.Contents.Add(_span);
sub.ContentStrings.Add(_span.OuterXml);
}
}
}
else
{
var id = _bgImg.Replace("#", "");
if (imageDic.TryGetValue(id, out var value))
{
var _span = new XmlDocument().CreateElement("span");
_span.InnerText = $"Base64::{value}";
sub.Contents.Add(_span);
sub.ContentStrings.Add(_span.OuterXml);
}
}
// Check if one <p> has been splitted
var index = finalSubs.FindLastIndex(s => s.End == _begin && s.Region == _region && s.ContentStrings.SequenceEqual(sub.ContentStrings));
// Skip empty lines
if (sub.ContentStrings.Count <= 0)
continue;
// Extend <p> duration
if (index != -1)
finalSubs[index].End = sub.End;
else if (!finalSubs.Contains(sub))
finalSubs.Add(sub);
}
}
var dic = new Dictionary<string, string>();
foreach (var sub in finalSubs)
{
var key = $"{sub.Begin} --> {sub.End}";
foreach (var item in sub.Contents)
{
if (dic.ContainsKey(key))
{
if (item.GetAttribute("tts:fontStyle") == "italic" || item.GetAttribute("tts:fontStyle") == "oblique")
dic[key] = $"{dic[key]}\r\n<i>{GetTextFromElement(item)}</i>";
else
dic[key] = $"{dic[key]}\r\n{GetTextFromElement(item)}";
}
else
{
if (item.GetAttribute("tts:fontStyle") == "italic" || item.GetAttribute("tts:fontStyle") == "oblique")
dic.Add(key, $"<i>{GetTextFromElement(item)}</i>");
else
dic.Add(key, GetTextFromElement(item));
}
}
}
var vtt = new StringBuilder();
vtt.AppendLine("WEBVTT");
foreach (var item in dic)
{
vtt.AppendLine(item.Key);
vtt.AppendLine(item.Value);
vtt.AppendLine();
}
return WebVttSub.Parse(vtt.ToString(), baseTimestamp);
}
}

213
src/N_m3u8DL-RE.Parser/Mp4/MP4VttUtil.cs Normal file
View File

@ -0,0 +1,213 @@
using N_m3u8DL_RE.Common.Entity;
using System.Text;
namespace Mp4SubtitleParser;
public static class MP4VttUtil
{
public static (bool, uint) CheckInit(byte[] data)
{
uint timescale = 0;
bool sawWVTT = false;
// parse init
new MP4Parser()
.Box("moov", MP4Parser.Children)
.Box("trak", MP4Parser.Children)
.Box("mdia", MP4Parser.Children)
.FullBox("mdhd", box =>
{
if (box.Version is not (0 or 1))
throw new Exception("MDHD version can only be 0 or 1");
timescale = MP4Parser.ParseMDHD(box.Reader, box.Version);
})
.Box("minf", MP4Parser.Children)
.Box("stbl", MP4Parser.Children)
.FullBox("stsd", MP4Parser.SampleDescription)
.Box("wvtt", _ => {
// A valid vtt init segment, though we have no actual subtitles yet.
sawWVTT = true;
})
.Parse(data);
return (sawWVTT, timescale);
}
public static WebVttSub ExtractSub(IEnumerable<string> files, uint timescale)
{
if (timescale == 0)
throw new Exception("Missing timescale for VTT content!");
List<SubCue> cues = [];
foreach (var item in files)
{
var dataSeg = File.ReadAllBytes(item);
bool sawTFDT = false;
bool sawTRUN = false;
bool sawMDAT = false;
byte[]? rawPayload = null;
ulong baseTime = 0;
ulong defaultDuration = 0;
List<Sample> presentations = [];
// parse media
new MP4Parser()
.Box("moof", MP4Parser.Children)
.Box("traf", MP4Parser.Children)
.FullBox("tfdt", box =>
{
sawTFDT = true;
if (box.Version is not (0 or 1))
throw new Exception("TFDT version can only be 0 or 1");
baseTime = MP4Parser.ParseTFDT(box.Reader, box.Version);
})
.FullBox("tfhd", box =>
{
if (box.Flags == 1000)
throw new Exception("A TFHD box should have a valid flags value");
defaultDuration = MP4Parser.ParseTFHD(box.Reader, box.Flags).DefaultSampleDuration;
})
.FullBox("trun", box =>
{
sawTRUN = true;
if (box.Version == 1000)
throw new Exception("A TRUN box should have a valid version value");
if (box.Flags == 1000)
throw new Exception("A TRUN box should have a valid flags value");
presentations = MP4Parser.ParseTRUN(box.Reader, box.Version, box.Flags).SampleData;
})
.Box("mdat", MP4Parser.AllData(data =>
{
if (sawMDAT)
throw new Exception("VTT cues in mp4 with multiple MDAT are not currently supported");
sawMDAT = true;
rawPayload = data;
}))
.Parse(dataSeg,/* partialOkay= */ false);
if (!sawMDAT && !sawTFDT && !sawTRUN)
{
throw new Exception("A required box is missing");
}
var currentTime = baseTime;
var reader = new BinaryReader2(new MemoryStream(rawPayload!));
foreach (var presentation in presentations)
{
var duration = presentation.SampleDuration == 0 ? defaultDuration : presentation.SampleDuration;
var startTime = presentation.SampleCompositionTimeOffset != 0 ?
baseTime + presentation.SampleCompositionTimeOffset :
currentTime;
currentTime = startTime + duration;
var totalSize = 0;
do
{
// Read the payload size.
var payloadSize = (int)reader.ReadUInt32();
totalSize += payloadSize;
// Skip the type.
var payloadType = reader.ReadUInt32();
var payloadName = MP4Parser.TypeToString(payloadType);
// Read the data payload.
byte[]? payload = null;
if (payloadName == "vttc")
{
if (payloadSize > 8)
{
payload = reader.ReadBytes(payloadSize - 8);
}
}
else if (payloadName == "vtte")
{
// It's a vtte, which is a vtt cue that is empty. Ignore any data that
// does exist.
reader.ReadBytes(payloadSize - 8);
}
else
{
Console.WriteLine($"Unknown box {payloadName}! Skipping!");
reader.ReadBytes(payloadSize - 8);
}
if (duration != 0)
{
if (payload != null)
{
var cue = ParseVTTC(
payload,
0 + (double)startTime / timescale,
0 + (double)currentTime / timescale);
// Check if same subtitle has been splitted
if (cue != null)
{
var index = cues.FindLastIndex(s => s.EndTime == cue.StartTime && s.Settings == cue.Settings && s.Payload == cue.Payload);
if (index != -1)
{
cues[index].EndTime = cue.EndTime;
}
else
{
cues.Add(cue);
}
}
}
}
else
{
throw new Exception("WVTT sample duration unknown, and no default found!");
}
if (!(presentation.SampleSize == 0 || totalSize <= presentation.SampleSize))
{
throw new Exception("The samples do not fit evenly into the sample sizes given in the TRUN box!");
}
} while (presentation.SampleSize != 0 && (totalSize < presentation.SampleSize));
if (reader.HasMoreData())
{
// throw new Exception("MDAT which contain VTT cues and non-VTT data are not currently supported!");
}
}
}
if (cues.Count > 0)
{
return new WebVttSub() { Cues = cues };
}
return new WebVttSub();
}
private static SubCue? ParseVTTC(byte[] data, double startTime, double endTime)
{
string payload = string.Empty;
string id = string.Empty;
string settings = string.Empty;
new MP4Parser()
.Box("payl", MP4Parser.AllData(data =>
{
payload = Encoding.UTF8.GetString(data);
}))
.Box("iden", MP4Parser.AllData(data =>
{
id = Encoding.UTF8.GetString(data);
}))
.Box("sttg", MP4Parser.AllData(data =>
{
settings = Encoding.UTF8.GetString(data);
}))
.Parse(data);
if (!string.IsNullOrEmpty(payload))
{
return new SubCue() { StartTime = TimeSpan.FromSeconds(startTime), EndTime = TimeSpan.FromSeconds(endTime), Payload = payload, Settings = settings };
}
return null;
}
}

869
src/N_m3u8DL-RE.Parser/Mp4/MSSMoovProcessor.cs Normal file
View File

@ -0,0 +1,869 @@
using Mp4SubtitleParser;
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Util;
using System.Text;
using System.Text.RegularExpressions;
// https://github.com/canalplus/rx-player/blob/48d1f845064cea5c5a3546d2c53b1855c2be149d/src/parsers/manifest/smooth/get_codecs.ts
// https://github.dev/Dash-Industry-Forum/dash.js/blob/2aad3e79079b4de0bcd961ce6b4957103d98a621/src/mss/MssFragmentMoovProcessor.js
// https://github.com/yt-dlp/yt-dlp/blob/3639df54c3298e35b5ae2a96a25bc4d3c38950d0/yt_dlp/downloader/ism.py
// https://github.com/google/ExoPlayer/blob/a9444c880230d2c2c79097e89259ce0b9f80b87d/library/extractor/src/main/java/com/google/android/exoplayer2/video/HevcConfig.java#L38
// https://github.com/sannies/mp4parser/blob/master/isoparser/src/main/java/org/mp4parser/boxes/iso14496/part15/HevcDecoderConfigurationRecord.java
namespace N_m3u8DL_RE.Parser.Mp4;
public partial class MSSMoovProcessor
{
[GeneratedRegex(@"\<KID\>(.*?)\<")]
private static partial Regex KIDRegex();
private static string StartCode = "00000001";
private StreamSpec StreamSpec;
private int TrackId = 2;
private string FourCC;
private string CodecPrivateData;
private int Timesacle;
private long Duration;
private string Language => StreamSpec.Language ?? "und";
private int Width => int.Parse((StreamSpec.Resolution ?? "0x0").Split('x').First());
private int Height => int.Parse((StreamSpec.Resolution ?? "0x0").Split('x').Last());
private string StreamType;
private int Channels;
private int BitsPerSample;
private int SamplingRate;
private int NalUnitLengthField;
private long CreationTime = DateTimeOffset.UtcNow.ToUnixTimeSeconds();
private bool IsProtection;
private string ProtectionSystemId;
private string ProtectionData;
private string? ProtecitonKID;
private string? ProtecitonKID_PR;
private byte[] UnityMatrix
{
get
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteInt(0x10000);
writer.WriteInt(0);
writer.WriteInt(0);
writer.WriteInt(0);
writer.WriteInt(0x10000);
writer.WriteInt(0);
writer.WriteInt(0);
writer.WriteInt(0);
writer.WriteInt(0x40000000);
return stream.ToArray();
}
}
private static byte TRACK_ENABLED = 0x1;
private static byte TRACK_IN_MOVIE = 0x2;
private static byte TRACK_IN_PREVIEW = 0x4;
private static byte SELF_CONTAINED = 0x1;
private static List<string> SupportedFourCC =
["HVC1", "HEV1", "AACL", "AACH", "EC-3", "H264", "AVC1", "DAVC", "AVC1", "TTML", "DVHE", "DVH1"];
public MSSMoovProcessor(StreamSpec streamSpec)
{
this.StreamSpec = streamSpec;
var data = streamSpec.MSSData!;
this.NalUnitLengthField = data.NalUnitLengthField;
this.CodecPrivateData = data.CodecPrivateData;
this.FourCC = data.FourCC;
this.Timesacle = data.Timesacle;
this.Duration = data.Duration;
this.StreamType = data.Type;
this.Channels = data.Channels;
this.SamplingRate = data.SamplingRate;
this.BitsPerSample = data.BitsPerSample;
this.IsProtection = data.IsProtection;
this.ProtectionData = data.ProtectionData;
this.ProtectionSystemId = data.ProtectionSystemID;
// 需要手动生成CodecPrivateData
if (string.IsNullOrEmpty(CodecPrivateData))
{
GenCodecPrivateDataForAAC();
}
// 解析KID
if (IsProtection)
{
ExtractKID();
}
}
private static string[] HEVC_GENERAL_PROFILE_SPACE_STRINGS = ["", "A", "B", "C"];
private int SamplingFrequencyIndex(int samplingRate) => samplingRate switch
{
96000 => 0x0,
88200 => 0x1,
64000 => 0x2,
48000 => 0x3,
44100 => 0x4,
32000 => 0x5,
24000 => 0x6,
22050 => 0x7,
16000 => 0x8,
12000 => 0x9,
11025 => 0xA,
8000 => 0xB,
7350 => 0xC,
_ => 0x0
};
private void GenCodecPrivateDataForAAC()
{
var objectType = 0x02; // AAC Main Low Complexity => object Type = 2
var indexFreq = SamplingFrequencyIndex(SamplingRate);
if (FourCC == "AACH")
{
// 4 bytes : XXXXX XXXX XXXX XXXX XXXXX XXX XXXXXXX
// ' ObjectType' 'Freq Index' 'Channels value' 'Extens Sampl Freq' 'ObjectType' 'GAS' 'alignment = 0'
objectType = 0x05; // High Efficiency AAC Profile = object Type = 5 SBR
var codecPrivateData = new byte[4];
var extensionSamplingFrequencyIndex = SamplingFrequencyIndex(SamplingRate * 2); // in HE AAC Extension Sampling frequence
// equals to SamplingRate*2
// Freq Index is present for 3 bits in the first byte, last bit is in the second
codecPrivateData[0] = (byte)((objectType << 3) | (indexFreq >> 1));
codecPrivateData[1] = (byte)((indexFreq << 7) | (Channels << 3) | (extensionSamplingFrequencyIndex >> 1));
codecPrivateData[2] = (byte)((extensionSamplingFrequencyIndex << 7) | (0x02 << 2)); // origin object type equals to 2 => AAC Main Low Complexity
codecPrivateData[3] = 0x0; // alignment bits
var arr16 = new ushort[2];
arr16[0] = (ushort)((codecPrivateData[0] << 8) + codecPrivateData[1]);
arr16[1] = (ushort)((codecPrivateData[2] << 8) + codecPrivateData[3]);
// convert decimal to hex value
this.CodecPrivateData = HexUtil.BytesToHex(BitConverter.GetBytes(arr16[0])).PadLeft(16, '0');
this.CodecPrivateData += HexUtil.BytesToHex(BitConverter.GetBytes(arr16[1])).PadLeft(16, '0');
}
else if (FourCC.StartsWith("AAC"))
{
// 2 bytes : XXXXX XXXX XXXX XXX
// ' ObjectType' 'Freq Index' 'Channels value' 'GAS = 000'
var codecPrivateData = new byte[2];
// Freq Index is present for 3 bits in the first byte, last bit is in the second
codecPrivateData[0] = (byte)((objectType << 3) | (indexFreq >> 1));
codecPrivateData[1] = (byte)((indexFreq << 7) | Channels << 3);
// put the 2 bytes in an 16 bits array
var arr16 = new ushort[1];
arr16[0] = (ushort)((codecPrivateData[0] << 8) + codecPrivateData[1]);
// convert decimal to hex value
this.CodecPrivateData = HexUtil.BytesToHex(BitConverter.GetBytes(arr16[0])).PadLeft(16, '0');
}
}
private void ExtractKID()
{
// playready
if (ProtectionSystemId.ToUpper() == "9A04F079-9840-4286-AB92-E65BE0885F95")
{
var bytes = HexUtil.HexToBytes(ProtectionData.Replace("00", ""));
var text = Encoding.ASCII.GetString(bytes);
var kidBytes = Convert.FromBase64String(KIDRegex().Match(text).Groups[1].Value);
// save kid for playready
this.ProtecitonKID_PR = HexUtil.BytesToHex(kidBytes);
// fix byte order
var reverse1 = new[] { kidBytes[3], kidBytes[2], kidBytes[1], kidBytes[0] };
var reverse2 = new[] { kidBytes[5], kidBytes[4], kidBytes[7], kidBytes[6] };
Array.Copy(reverse1, 0, kidBytes, 0, reverse1.Length);
Array.Copy(reverse2, 0, kidBytes, 4, reverse1.Length);
this.ProtecitonKID = HexUtil.BytesToHex(kidBytes);
}
// widevine
else if (ProtectionSystemId.ToUpper() == "EDEF8BA9-79D6-4ACE-A3C8-27DCD51D21ED")
{
throw new NotSupportedException();
}
}
public static bool CanHandle(string fourCC) => SupportedFourCC.Contains(fourCC);
private byte[] Box(string boxType, byte[] payload)
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteUInt(8 + (uint)payload.Length);
writer.Write(boxType);
writer.Write(payload);
return stream.ToArray();
}
private byte[] FullBox(string boxType, byte version, uint flags, byte[] payload)
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.Write(version);
writer.WriteUInt(flags, offset: 1);
writer.Write(payload);
return Box(boxType, stream.ToArray());
}
private byte[] GenSinf(string codec)
{
var frmaBox = Box("frma", Encoding.ASCII.GetBytes(codec));
var sinfPayload = new List<byte>();
sinfPayload.AddRange(frmaBox);
var schmPayload = new List<byte>();
schmPayload.AddRange(Encoding.ASCII.GetBytes("cenc")); // scheme_type 'cenc' => common encryption
schmPayload.AddRange([0, 1, 0, 0]); // scheme_version Major version 1, Minor version 0
var schmBox = FullBox("schm", 0, 0, schmPayload.ToArray());
sinfPayload.AddRange(schmBox);
var tencPayload = new List<byte>();
tencPayload.AddRange([0, 0]);
tencPayload.Add(0x1); // default_IsProtected
tencPayload.Add(0x8); // default_Per_Sample_IV_size
tencPayload.AddRange(HexUtil.HexToBytes(ProtecitonKID)); // default_KID
// tencPayload.Add(0x8);// default_constant_IV_size
// tencPayload.AddRange(new byte[8]);// default_constant_IV
var tencBox = FullBox("tenc", 0, 0, tencPayload.ToArray());
var schiBox = Box("schi", tencBox);
sinfPayload.AddRange(schiBox);
var sinfBox = Box("sinf", sinfPayload.ToArray());
return sinfBox;
}
private byte[] GenFtyp()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.Write("isml"); // major brand
writer.WriteUInt(1); // minor version
writer.Write("iso5"); // compatible brand
writer.Write("iso6"); // compatible brand
writer.Write("piff"); // compatible brand
writer.Write("msdh"); // compatible brand
return Box("ftyp", stream.ToArray());
}
private byte[] GenMvhd()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteULong(CreationTime); // creation_time
writer.WriteULong(CreationTime); // modification_time
writer.WriteUInt(Timesacle); // timescale
writer.WriteULong(Duration); // duration
writer.WriteUShort(1, padding: 2); // rate
writer.WriteByte(1, padding: 1); // volume
writer.WriteUShort(0); // reserved
writer.WriteUInt(0);
writer.WriteUInt(0);
writer.Write(UnityMatrix);
writer.WriteUInt(0); // pre defined
writer.WriteUInt(0);
writer.WriteUInt(0);
writer.WriteUInt(0);
writer.WriteUInt(0);
writer.WriteUInt(0);
writer.WriteUInt(0xffffffff); // next track id
return FullBox("mvhd", 1, 0, stream.ToArray());
}
private byte[] GenTkhd()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteULong(CreationTime); // creation_time
writer.WriteULong(CreationTime); // modification_time
writer.WriteUInt(TrackId); // track id
writer.WriteUInt(0); // reserved
writer.WriteULong(Duration); // duration
writer.WriteUInt(0); // reserved
writer.WriteUInt(0);
writer.WriteShort(0); // layer
writer.WriteShort(0); // alternate group
writer.WriteByte(StreamType == "audio" ? (byte)1 : (byte)0, padding: 1); // volume
writer.WriteUShort(0); // reserved
writer.Write(UnityMatrix);
writer.WriteUShort(Width, padding: 2); // width
writer.WriteUShort(Height, padding: 2); // height
return FullBox("tkhd", 1, (uint)TRACK_ENABLED | TRACK_IN_MOVIE | TRACK_IN_PREVIEW, stream.ToArray());
}
private byte[] GenMdhd()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteULong(CreationTime); // creation_time
writer.WriteULong(CreationTime); // modification_time
writer.WriteUInt(Timesacle); // timescale
writer.WriteULong(Duration); // duration
writer.WriteUShort((Language[0] - 0x60) << 10 | (Language[1] - 0x60) << 5 | (Language[2] - 0x60)); // language
writer.WriteUShort(0); // pre defined
return FullBox("mdhd", 1, 0, stream.ToArray());
}
private byte[] GenHdlr()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteUInt(0); // pre defined
if (StreamType == "audio") writer.Write("soun");
else if (StreamType == "video") writer.Write("vide");
else if (StreamType == "text") writer.Write("subt");
else throw new NotSupportedException();
writer.WriteUInt(0); // reserved
writer.WriteUInt(0);
writer.WriteUInt(0);
writer.Write($"{StreamSpec.GroupId ?? "RE Handler"}\0"); // name
return FullBox("hdlr", 0, 0, stream.ToArray());
}
private byte[] GenMinf()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
var minfPayload = new List<byte>();
if (StreamType == "audio")
{
var smhd = new List<byte>();
smhd.Add(0); smhd.Add(0); // balance
smhd.Add(0); smhd.Add(0); // reserved
minfPayload.AddRange(FullBox("smhd", 0, 0, smhd.ToArray())); // Sound Media Header
}
else if (StreamType == "video")
{
var vmhd = new List<byte>();
vmhd.Add(0); vmhd.Add(0); // graphics mode
vmhd.Add(0); vmhd.Add(0); vmhd.Add(0); vmhd.Add(0); vmhd.Add(0); vmhd.Add(0);// opcolor
minfPayload.AddRange(FullBox("vmhd", 0, 1, vmhd.ToArray())); // Video Media Header
}
else if (StreamType == "text")
{
minfPayload.AddRange(FullBox("sthd", 0, 0, [])); // Subtitle Media Header
}
else
{
throw new NotSupportedException();
}
var drefPayload = new List<byte>();
drefPayload.Add(0); drefPayload.Add(0); drefPayload.Add(0); drefPayload.Add(1); // entry count
drefPayload.AddRange(FullBox("url ", 0, SELF_CONTAINED, [])); // Data Entry URL Box
var dinfPayload = FullBox("dref", 0, 0, drefPayload.ToArray()); // Data Reference Box
minfPayload.AddRange(Box("dinf", dinfPayload.ToArray())); // Data Information Box
return minfPayload.ToArray();
}
private byte[] GenEsds(byte[] audioSpecificConfig)
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
// ESDS length = esds box header length (= 12) +
// ES_Descriptor header length (= 5) +
// DecoderConfigDescriptor header length (= 15) +
// decoderSpecificInfo header length (= 2) +
// AudioSpecificConfig length (= codecPrivateData length)
// esdsLength = 34 + len(audioSpecificConfig)
// ES_Descriptor (see ISO/IEC 14496-1 (Systems))
writer.WriteByte(0x03); // tag = 0x03 (ES_DescrTag)
writer.WriteByte((byte)(20 + audioSpecificConfig.Length)); // size
writer.WriteByte((byte)((TrackId & 0xFF00) >> 8)); // ES_ID = track_id
writer.WriteByte((byte)(TrackId & 0x00FF));
writer.WriteByte(0); // flags and streamPriority
// DecoderConfigDescriptor (see ISO/IEC 14496-1 (Systems))
writer.WriteByte(0x04); // tag = 0x04 (DecoderConfigDescrTag)
writer.WriteByte((byte)(15 + audioSpecificConfig.Length)); // size
writer.WriteByte(0x40); // objectTypeIndication = 0x40 (MPEG-4 AAC)
writer.WriteByte((0x05 << 2) | (0 << 1) | 1); // reserved = 1
writer.WriteByte(0xFF); // buffersizeDB = undefined
writer.WriteByte(0xFF);
writer.WriteByte(0xFF);
var bandwidth = StreamSpec.Bandwidth!;
writer.WriteByte((byte)((bandwidth & 0xFF000000) >> 24)); // maxBitrate
writer.WriteByte((byte)((bandwidth & 0x00FF0000) >> 16));
writer.WriteByte((byte)((bandwidth & 0x0000FF00) >> 8));
writer.WriteByte((byte)(bandwidth & 0x000000FF));
writer.WriteByte((byte)((bandwidth & 0xFF000000) >> 24)); // avgbitrate
writer.WriteByte((byte)((bandwidth & 0x00FF0000) >> 16));
writer.WriteByte((byte)((bandwidth & 0x0000FF00) >> 8));
writer.WriteByte((byte)(bandwidth & 0x000000FF));
// DecoderSpecificInfo (see ISO/IEC 14496-1 (Systems))
writer.WriteByte(0x05); // tag = 0x05 (DecSpecificInfoTag)
writer.WriteByte((byte)audioSpecificConfig.Length); // size
writer.Write(audioSpecificConfig); // AudioSpecificConfig bytes
return FullBox("esds", 0, 0, stream.ToArray());
}
private byte[] GetSampleEntryBox()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteByte(0); // reserved
writer.WriteByte(0);
writer.WriteByte(0);
writer.WriteByte(0);
writer.WriteByte(0);
writer.WriteByte(0);
writer.WriteUShort(1); // data reference index
if (StreamType == "audio")
{
writer.WriteUInt(0); // reserved2
writer.WriteUInt(0);
writer.WriteUShort(Channels); // channels
writer.WriteUShort(BitsPerSample); // bits_per_sample
writer.WriteUShort(0); // pre defined
writer.WriteUShort(0); // reserved3
writer.WriteUShort(SamplingRate, padding: 2); // sampling_rate
var audioSpecificConfig = HexUtil.HexToBytes(CodecPrivateData);
var esdsBox = GenEsds(audioSpecificConfig);
writer.Write(esdsBox);
if (FourCC.StartsWith("AAC"))
{
if (IsProtection)
{
var sinfBox = GenSinf("mp4a");
writer.Write(sinfBox);
return Box("enca", stream.ToArray()); // Encrypted Audio
}
return Box("mp4a", stream.ToArray());
}
if (FourCC == "EC-3")
{
if (IsProtection)
{
var sinfBox = GenSinf("ec-3");
writer.Write(sinfBox);
return Box("enca", stream.ToArray()); // Encrypted Audio
}
return Box("ec-3", stream.ToArray());
}
}
else if (StreamType == "video")
{
writer.WriteUShort(0); // pre defined
writer.WriteUShort(0); // reserved
writer.WriteUInt(0); // pre defined
writer.WriteUInt(0);
writer.WriteUInt(0);
writer.WriteUShort(Width); // width
writer.WriteUShort(Height); // height
writer.WriteUShort(0x48, padding: 2); // horiz resolution 72 dpi
writer.WriteUShort(0x48, padding: 2); // vert resolution 72 dpi
writer.WriteUInt(0); // reserved
writer.WriteUShort(1); // frame count
for (int i = 0; i < 32; i++) // compressor name
{
writer.WriteByte(0);
}
writer.WriteUShort(0x18); // depth
writer.WriteUShort(65535); // pre defined
var codecPrivateData = HexUtil.HexToBytes(CodecPrivateData);
if (FourCC is "H264" or "AVC1" or "DAVC")
{
var arr = CodecPrivateData.Split([StartCode], StringSplitOptions.RemoveEmptyEntries);
var sps = HexUtil.HexToBytes(arr.First(x => (HexUtil.HexToBytes(x[0..2])[0] & 0x1F) == 7));
var pps = HexUtil.HexToBytes(arr.First(x => (HexUtil.HexToBytes(x[0..2])[0] & 0x1F) == 8));
// make avcC
var avcC = GetAvcC(sps, pps);
writer.Write(avcC);
if (IsProtection)
{
var sinfBox = GenSinf("avc1");
writer.Write(sinfBox);
return Box("encv", stream.ToArray()); // Encrypted Video
}
return Box("avc1", stream.ToArray()); // AVC Simple Entry
}
if (FourCC is "HVC1" or "HEV1")
{
var arr = CodecPrivateData.Split([StartCode], StringSplitOptions.RemoveEmptyEntries);
var vps = HexUtil.HexToBytes(arr.First(x => (HexUtil.HexToBytes(x[0..2])[0] >> 1) == 0x20));
var sps = HexUtil.HexToBytes(arr.First(x => (HexUtil.HexToBytes(x[0..2])[0] >> 1) == 0x21));
var pps = HexUtil.HexToBytes(arr.First(x => (HexUtil.HexToBytes(x[0..2])[0] >> 1) == 0x22));
// make hvcC
var hvcC = GetHvcC(sps, pps, vps);
writer.Write(hvcC);
if (IsProtection)
{
var sinfBox = GenSinf("hvc1");
writer.Write(sinfBox);
return Box("encv", stream.ToArray()); // Encrypted Video
}
return Box("hvc1", stream.ToArray()); // HEVC Simple Entry
}
// 杜比视界也按照hevc处理
if (FourCC is "DVHE" or "DVH1")
{
var arr = CodecPrivateData.Split([StartCode], StringSplitOptions.RemoveEmptyEntries);
var vps = HexUtil.HexToBytes(arr.First(x => (HexUtil.HexToBytes(x[0..2])[0] >> 1) == 0x20));
var sps = HexUtil.HexToBytes(arr.First(x => (HexUtil.HexToBytes(x[0..2])[0] >> 1) == 0x21));
var pps = HexUtil.HexToBytes(arr.First(x => (HexUtil.HexToBytes(x[0..2])[0] >> 1) == 0x22));
// make hvcC
var hvcC = GetHvcC(sps, pps, vps, "dvh1");
writer.Write(hvcC);
if (IsProtection)
{
var sinfBox = GenSinf("dvh1");
writer.Write(sinfBox);
return Box("encv", stream.ToArray()); // Encrypted Video
}
return Box("dvh1", stream.ToArray()); // HEVC Simple Entry
}
throw new NotSupportedException();
}
else if (StreamType == "text")
{
if (FourCC == "TTML")
{
writer.Write("http://www.w3.org/ns/ttml\0"); // namespace
writer.Write("\0"); // schema location
writer.Write("\0"); // auxilary mime types(??)
return Box("stpp", stream.ToArray()); // TTML Simple Entry
}
throw new NotSupportedException();
}
else
{
throw new NotSupportedException();
}
throw new NotSupportedException();
}
private byte[] GetAvcC(byte[] sps, byte[] pps)
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteByte(1); // configuration version
writer.Write(sps[1..4]); // avc profile indication + profile compatibility + avc level indication
writer.WriteByte((byte)(0xfc | (NalUnitLengthField - 1))); // complete representation (1) + reserved (11111) + length size minus one
writer.WriteByte(1); // reserved (0) + number of sps (0000001)
writer.WriteUShort(sps.Length);
writer.Write(sps);
writer.WriteByte(1); // number of pps
writer.WriteUShort(pps.Length);
writer.Write(pps);
return Box("avcC", stream.ToArray()); // AVC Decoder Configuration Record
}
private byte[] GetHvcC(byte[] sps, byte[] pps, byte[] vps, string code = "hvc1")
{
var oriSps = new List<byte>(sps);
// https://www.itu.int/rec/dologin.asp?lang=f&id=T-REC-H.265-201504-S!!PDF-E&type=items
// Read generalProfileSpace, generalTierFlag, generalProfileIdc,
// generalProfileCompatibilityFlags, constraintBytes, generalLevelIdc
// from sps
var encList = new List<byte>();
/**
* 处理payload, 有00 00 03 0,1,2,3的情况 统一换成00 00 XX 即丢弃03
* 注意：此处采用的逻辑是直接简单粗暴地判断列表末尾3字节，如果是0x000003就删掉最后的0x03，可能会导致以下情况
* 00 00 03 03 03 03 03 01 会被直接处理成 => 00 00 01
* 此处经过测试只有直接跳过才正常，如果处理成 00 00 03 03 03 03 01 是有问题的
*
* 测试的数据如下：
* 原始：42 01 01 01 60 00 00 03 00 90 00 00 03 00 00 03 00 96 a0 01 e0 20 06 61 65 95 9a 49 30 bf fc 0c 7c 0c 81 a8 08 08 08 20 00 00 03 00 20 00 00 03 03 01
* 处理后：42 01 01 01 60 00 00 00 90 00 00 00 00 00 96 A0 01 E0 20 06 61 65 95 9A 49 30 BF FC 0C 7C 0C 81 A8 08 08 08 20 00 00 00 20 00 00 01
*/
using (var _reader = new BinaryReader(new MemoryStream(sps)))
{
while (_reader.BaseStream.Position < _reader.BaseStream.Length)
{
encList.Add(_reader.ReadByte());
if (encList is [.., 0x00, 0x00, 0x03])
{
encList.RemoveAt(encList.Count - 1);
}
}
}
sps = encList.ToArray();
using var reader = new BinaryReader2(new MemoryStream(sps));
reader.ReadBytes(2); // Skip 2 bytes unit header
var firstByte = reader.ReadByte();
var maxSubLayersMinus1 = (firstByte & 0xe) >> 1;
var nextByte = reader.ReadByte();
var generalProfileSpace = (nextByte & 0xc0) >> 6;
var generalTierFlag = (nextByte & 0x20) >> 5;
var generalProfileIdc = nextByte & 0x1f;
var generalProfileCompatibilityFlags = reader.ReadUInt32();
var constraintBytes = reader.ReadBytes(6);
var generalLevelIdc = reader.ReadByte();
/*var skipBit = 0;
for (int i = 0; i < maxSubLayersMinus1; i++)
{
skipBit += 2; // sub_layer_profile_present_flag sub_layer_level_present_flag
}
if (maxSubLayersMinus1 > 0)
{
for (int i = maxSubLayersMinus1; i < 8; i++)
{
skipBit += 2; // reserved_zero_2bits
}
}
for (int i = 0; i < maxSubLayersMinus1; i++)
{
skipBit += 2; // sub_layer_profile_present_flag sub_layer_level_present_flag
}*/
// 生成编码信息
var codecs = code +
$".{HEVC_GENERAL_PROFILE_SPACE_STRINGS[generalProfileSpace]}{generalProfileIdc}" +
$".{Convert.ToString(generalProfileCompatibilityFlags, 16)}" +
$".{(generalTierFlag == 1 ? 'H' : 'L')}{generalLevelIdc}" +
$".{HexUtil.BytesToHex(constraintBytes.Where(b => b != 0).ToArray())}";
StreamSpec.Codecs = codecs;
///////////////////////
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
// var reserved1 = 0xF;
writer.WriteByte(1); // configuration version
writer.WriteByte((byte)((generalProfileSpace << 6) + (generalTierFlag == 1 ? 0x20 : 0) | generalProfileIdc)); // general_profile_space + general_tier_flag + general_profile_idc
writer.WriteUInt(generalProfileCompatibilityFlags); // general_profile_compatibility_flags
writer.Write(constraintBytes); // general_constraint_indicator_flags
writer.WriteByte((byte)generalProfileIdc); // general_level_idc
writer.WriteUShort(0xf000); // reserved + min_spatial_segmentation_idc
writer.WriteByte(0xfc); // reserved + parallelismType
writer.WriteByte(0 | 0xfc); // reserved + chromaFormat
writer.WriteByte(0 | 0xf8); // reserved + bitDepthLumaMinus8
writer.WriteByte(0 | 0xf8); // reserved + bitDepthChromaMinus8
writer.WriteUShort(0); // avgFrameRate
writer.WriteByte((byte)(0 << 6 | 0 << 3 | 0 << 2 | (NalUnitLengthField - 1))); // constantFrameRate + numTemporalLayers + temporalIdNested + lengthSizeMinusOne
writer.WriteByte(0x03); // numOfArrays (vps sps pps)
sps = oriSps.ToArray();
writer.WriteByte(0x20); // array_completeness + reserved + NAL_unit_type
writer.WriteUShort(1); // numNalus
writer.WriteUShort(vps.Length);
writer.Write(vps);
writer.WriteByte(0x21);
writer.WriteUShort(1); // numNalus
writer.WriteUShort(sps.Length);
writer.Write(sps);
writer.WriteByte(0x22);
writer.WriteUShort(1); // numNalus
writer.WriteUShort(pps.Length);
writer.Write(pps);
return Box("hvcC", stream.ToArray()); // HEVC Decoder Configuration Record
}
private byte[] GetStsd()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteUInt(1); // entry count
var sampleEntryData = GetSampleEntryBox();
writer.Write(sampleEntryData);
return stream.ToArray();
}
private byte[] GetMehd()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteULong(Duration);
return FullBox("mehd", 1, 0, stream.ToArray()); // Movie Extends Header Box
}
private byte[] GetTrex()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
writer.WriteUInt(TrackId); // track id
writer.WriteUInt(1); // default sample description index
writer.WriteUInt(0); // default sample duration
writer.WriteUInt(0); // default sample size
writer.WriteUInt(0); // default sample flags
return FullBox("trex", 0, 0, stream.ToArray()); // Track Extends Box
}
private byte[] GenPsshBoxForPlayReady()
{
using var _stream = new MemoryStream();
using var _writer = new BinaryWriter2(_stream);
var sysIdData = HexUtil.HexToBytes(ProtectionSystemId.Replace("-", ""));
var psshData = HexUtil.HexToBytes(ProtectionData);
_writer.Write(sysIdData); // SystemID 16 bytes
_writer.WriteUInt(psshData.Length); // Size of Data 4 bytes
_writer.Write(psshData); // Data
var psshBox = FullBox("pssh", 0, 0, _stream.ToArray());
return psshBox;
}
private byte[] GenPsshBoxForWideVine()
{
using var _stream = new MemoryStream();
using var _writer = new BinaryWriter2(_stream);
var sysIdData = HexUtil.HexToBytes("edef8ba9-79d6-4ace-a3c8-27dcd51d21ed".Replace("-", ""));
// var kid = HexUtil.HexToBytes(ProtecitonKID);
_writer.Write(sysIdData); // SystemID 16 bytes
var psshData = HexUtil.HexToBytes($"08011210{ProtecitonKID}1A046E647265220400000000");
_writer.WriteUInt(psshData.Length); // Size of Data 4 bytes
_writer.Write(psshData); // Data
var psshBox = FullBox("pssh", 0, 0, _stream.ToArray());
return psshBox;
}
private byte[] GenMoof()
{
using var stream = new MemoryStream();
using var writer = new BinaryWriter2(stream);
// make senc
writer.WriteUInt(1); // sample_count
writer.Write(new byte[8]); // 8 bytes IV
var sencBox = FullBox("senc", 1, 0, stream.ToArray());
var moofBox = Box("moof", sencBox); // Movie Extends Box
return moofBox;
}
public byte[] GenHeader(byte[] firstSegment)
{
new MP4Parser()
.Box("moof", MP4Parser.Children)
.Box("traf", MP4Parser.Children)
.FullBox("tfhd", box =>
{
TrackId = (int)box.Reader.ReadUInt32();
})
.Parse(firstSegment);
return GenHeader();
}
public byte[] GenHeader()
{
using var stream = new MemoryStream();
var ftyp = GenFtyp(); // File Type Box
stream.Write(ftyp);
var moovPayload = GenMvhd(); // Movie Header Box
var trakPayload = GenTkhd(); // Track Header Box
var mdhdPayload = GenMdhd(); // Media Header Box
var hdlrPayload = GenHdlr(); // Handler Reference Box
var mdiaPayload = mdhdPayload.Concat(hdlrPayload).ToArray();
var minfPayload = GenMinf();
var sttsPayload = new byte[] { 0, 0, 0, 0 }; // entry count
var stblPayload = FullBox("stts", 0, 0, sttsPayload); // Decoding Time to Sample Box
var stscPayload = new byte[] { 0, 0, 0, 0 }; // entry count
var stscBox = FullBox("stsc", 0, 0, stscPayload); // Sample To Chunk Box
var stcoPayload = new byte[] { 0, 0, 0, 0 }; // entry count
var stcoBox = FullBox("stco", 0, 0, stcoPayload); // Chunk Offset Box
var stszPayload = new byte[] { 0, 0, 0, 0, 0, 0, 0, 0 }; // sample size, sample count
var stszBox = FullBox("stsz", 0, 0, stszPayload); // Sample Size Box
var stsdPayload = GetStsd();
var stsdBox = FullBox("stsd", 0, 0, stsdPayload); // Sample Description Box
stblPayload = stblPayload.Concat(stscBox).Concat(stcoBox).Concat(stszBox).Concat(stsdBox).ToArray();
var stblBox = Box("stbl", stblPayload); // Sample Table Box
minfPayload = minfPayload.Concat(stblBox).ToArray();
var minfBox = Box("minf", minfPayload); // Media Information Box
mdiaPayload = mdiaPayload.Concat(minfBox).ToArray();
var mdiaBox = Box("mdia", mdiaPayload); // Media Box
trakPayload = trakPayload.Concat(mdiaBox).ToArray();
var trakBox = Box("trak", trakPayload); // Track Box
moovPayload = moovPayload.Concat(trakBox).ToArray();
var mvexPayload = GetMehd();
var trexBox = GetTrex();
mvexPayload = mvexPayload.Concat(trexBox).ToArray();
var mvexBox = Box("mvex", mvexPayload); // Movie Extends Box
moovPayload = moovPayload.Concat(mvexBox).ToArray();
if (IsProtection)
{
var psshBox1 = GenPsshBoxForPlayReady();
var psshBox2 = GenPsshBoxForWideVine();
moovPayload = moovPayload.Concat(psshBox1).Concat(psshBox2).ToArray();
}
var moovBox = Box("moov", moovPayload); // Movie Box
stream.Write(moovBox);
// var moofBox = GenMoof(); // Movie Extends Box
// stream.Write(moofBox);
return stream.ToArray();
}
}

16
src/N_m3u8DL-RE.Parser/N_m3u8DL-RE.Parser.csproj Normal file
View File

@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>library</OutputType>
<TargetFramework>net9.0</TargetFramework>
<RootNamespace>N_m3u8DL_RE.Parser</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\N_m3u8DL-RE.Common\N_m3u8DL-RE.Common.csproj" />
</ItemGroup>
</Project>

10
src/N_m3u8DL-RE.Parser/Processor/ContentProcessor.cs Normal file
View File

@ -0,0 +1,10 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Parser.Config;
namespace N_m3u8DL_RE.Parser.Processor;
public abstract class ContentProcessor
{
public abstract bool CanProcess(ExtractorType extractorType, string rawText, ParserConfig parserConfig);
public abstract string Process(string rawText, ParserConfig parserConfig);
}

26
src/N_m3u8DL-RE.Parser/Processor/DASH/DefaultDASHContentProcessor.cs Normal file
View File

@ -0,0 +1,26 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Parser.Config;
namespace N_m3u8DL_RE.Parser.Processor.DASH;
/// <summary>
/// XG视频处理
/// </summary>
public class DefaultDASHContentProcessor : ContentProcessor
{
public override bool CanProcess(ExtractorType extractorType, string mpdContent, ParserConfig parserConfig)
{
if (extractorType != ExtractorType.MPEG_DASH) return false;
return mpdContent.Contains("<mas:") && !mpdContent.Contains("xmlns:mas");
}
public override string Process(string mpdContent, ParserConfig parserConfig)
{
Logger.Debug("Fix xigua mpd...");
mpdContent = mpdContent.Replace("<MPD ", "<MPD xmlns:mas=\"urn:marlin:mas:1-0:services:schemas:mpd\" ");
return mpdContent;
}
}

37
src/N_m3u8DL-RE.Parser/Processor/DefaultUrlProcessor.cs Normal file
View File

@ -0,0 +1,37 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Parser.Config;
using System.Web;
namespace N_m3u8DL_RE.Parser.Processor;
public class DefaultUrlProcessor : UrlProcessor
{
public override bool CanProcess(ExtractorType extractorType, string oriUrl, ParserConfig paserConfig) => paserConfig.AppendUrlParams;
public override string Process(string oriUrl, ParserConfig paserConfig)
{
if (!oriUrl.StartsWith("http")) return oriUrl;
var uriFromConfig = new Uri(paserConfig.Url);
var uriFromConfigQuery = HttpUtility.ParseQueryString(uriFromConfig.Query);
var oldUri = new Uri(oriUrl);
var newQuery = HttpUtility.ParseQueryString(oldUri.Query);
foreach (var item in uriFromConfigQuery.AllKeys)
{
if (newQuery.AllKeys.Contains(item))
newQuery.Set(item, uriFromConfigQuery.Get(item));
else
newQuery.Add(item, uriFromConfigQuery.Get(item));
}
if (string.IsNullOrEmpty(newQuery.ToString())) return oriUrl;
Logger.Debug("Before: " + oriUrl);
oriUrl = (oldUri.GetLeftPart(UriPartial.Path) + "?" + newQuery).TrimEnd('?');
Logger.Debug("After: " + oriUrl);
return oriUrl;
}
}

96
src/N_m3u8DL-RE.Parser/Processor/HLS/DefaultHLSContentProcessor.cs Normal file
View File

@ -0,0 +1,96 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Parser.Constants;
using System.Text.RegularExpressions;
namespace N_m3u8DL_RE.Parser.Processor.HLS;
public partial class DefaultHLSContentProcessor : ContentProcessor
{
[GeneratedRegex("#EXT-X-DISCONTINUITY\\s+#EXT-X-MAP:URI=\\\"(.*?)\\\",BYTERANGE=\\\"(.*?)\\\"")]
private static partial Regex YkDVRegex();
[GeneratedRegex("#EXT-X-MAP:URI=\\\".*?BUMPER/[\\s\\S]+?#EXT-X-DISCONTINUITY")]
private static partial Regex DNSPRegex();
[GeneratedRegex(@"#EXTINF:.*?,\s+.*BUMPER.*\s+?#EXT-X-DISCONTINUITY")]
private static partial Regex DNSPSubRegex();
[GeneratedRegex("(#EXTINF.*)(\\s+)(#EXT-X-KEY.*)")]
private static partial Regex OrderFixRegex();
[GeneratedRegex(@"#EXT-X-MAP.*\.apple\.com/")]
private static partial Regex ATVRegex();
[GeneratedRegex(@"(#EXT-X-KEY:[\s\S]*?)(#EXT-X-DISCONTINUITY|#EXT-X-ENDLIST)")]
private static partial Regex ATVRegex2();
public override bool CanProcess(ExtractorType extractorType, string rawText, ParserConfig parserConfig) => extractorType == ExtractorType.HLS;
public override string Process(string m3u8Content, ParserConfig parserConfig)
{
// 处理content以\r作为换行符的情况
if (m3u8Content.Contains('\r') && !m3u8Content.Contains('\n'))
{
m3u8Content = m3u8Content.Replace("\r", Environment.NewLine);
}
var m3u8Url = parserConfig.Url;
// YSP回放
if (m3u8Url.Contains("tlivecloud-playback-cdn.ysp.cctv.cn") && m3u8Url.Contains("endtime="))
{
m3u8Content += Environment.NewLine + HLSTags.ext_x_endlist;
}
// IMOOC
if (m3u8Url.Contains("imooc.com/"))
{
// M3u8Content = DecodeImooc.DecodeM3u8(M3u8Content);
}
// 针对YK #EXT-X-VERSION:7杜比视界片源修正
if (m3u8Content.Contains("#EXT-X-DISCONTINUITY") && m3u8Content.Contains("#EXT-X-MAP") && m3u8Content.Contains("ott.cibntv.net") && m3u8Content.Contains("ccode="))
{
var ykmap = YkDVRegex();
foreach (Match m in ykmap.Matches(m3u8Content))
{
m3u8Content = m3u8Content.Replace(m.Value, $"#EXTINF:0.000000,\n#EXT-X-BYTERANGE:{m.Groups[2].Value}\n{m.Groups[1].Value}");
}
}
// 针对Disney+修正
if (m3u8Content.Contains("#EXT-X-DISCONTINUITY") && m3u8Content.Contains("#EXT-X-MAP") && m3u8Url.Contains("media.dssott.com/"))
{
Regex ykmap = DNSPRegex();
if (ykmap.IsMatch(m3u8Content))
{
m3u8Content = m3u8Content.Replace(ykmap.Match(m3u8Content).Value, "#XXX");
}
}
// 针对Disney+字幕修正
if (m3u8Content.Contains("#EXT-X-DISCONTINUITY") && m3u8Content.Contains("seg_00000.vtt") && m3u8Url.Contains("media.dssott.com/"))
{
Regex ykmap = DNSPSubRegex();
if (ykmap.IsMatch(m3u8Content))
{
m3u8Content = m3u8Content.Replace(ykmap.Match(m3u8Content).Value, "#XXX");
}
}
// 针对AppleTv修正
if (m3u8Content.Contains("#EXT-X-DISCONTINUITY") && m3u8Content.Contains("#EXT-X-MAP") && (m3u8Url.Contains(".apple.com/") || ATVRegex().IsMatch(m3u8Content)))
{
// 只取加密部分即可
Regex ykmap = ATVRegex2();
if (ykmap.IsMatch(m3u8Content))
{
m3u8Content = "#EXTM3U\r\n" + ykmap.Match(m3u8Content).Groups[1].Value + "\r\n#EXT-X-ENDLIST";
}
}
// 修复#EXT-X-KEY与#EXTINF出现次序异常问题
var regex = OrderFixRegex();
if (regex.IsMatch(m3u8Content))
{
m3u8Content = regex.Replace(m3u8Content, "$3$2$1");
}
return m3u8Content;
}
}

110
src/N_m3u8DL-RE.Parser/Processor/HLS/DefaultHLSKeyProcessor.cs Normal file
View File

@ -0,0 +1,110 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Parser.Util;
using Spectre.Console;
namespace N_m3u8DL_RE.Parser.Processor.HLS;
public class DefaultHLSKeyProcessor : KeyProcessor
{
public override bool CanProcess(ExtractorType extractorType, string m3u8Url, string keyLine, string m3u8Content, ParserConfig paserConfig) => extractorType == ExtractorType.HLS;
public override EncryptInfo Process(string keyLine, string m3u8Url, string m3u8Content, ParserConfig parserConfig)
{
var iv = ParserUtil.GetAttribute(keyLine, "IV");
var method = ParserUtil.GetAttribute(keyLine, "METHOD");
var uri = ParserUtil.GetAttribute(keyLine, "URI");
Logger.Debug("METHOD:{},URI:{},IV:{}", method, uri, iv);
var encryptInfo = new EncryptInfo(method);
// IV
if (!string.IsNullOrEmpty(iv))
{
encryptInfo.IV = HexUtil.HexToBytes(iv);
}
// 自定义IV
if (parserConfig.CustomeIV is { Length: > 0 })
{
encryptInfo.IV = parserConfig.CustomeIV;
}
// KEY
try
{
if (parserConfig.CustomeKey is { Length: > 0 })
{
encryptInfo.Key = parserConfig.CustomeKey;
}
else if (uri.ToLower().StartsWith("base64:"))
{
encryptInfo.Key = Convert.FromBase64String(uri[7..]);
}
else if (uri.ToLower().StartsWith("data:;base64,"))
{
encryptInfo.Key = Convert.FromBase64String(uri[13..]);
}
else if (uri.ToLower().StartsWith("data:text/plain;base64,"))
{
encryptInfo.Key = Convert.FromBase64String(uri[23..]);
}
else if (File.Exists(uri))
{
encryptInfo.Key = File.ReadAllBytes(uri);
}
else if (!string.IsNullOrEmpty(uri))
{
var retryCount = parserConfig.KeyRetryCount;
var segUrl = PreProcessUrl(ParserUtil.CombineURL(m3u8Url, uri), parserConfig);
getHttpKey:
try
{
var bytes = HTTPUtil.GetBytesAsync(segUrl, parserConfig.Headers).Result;
encryptInfo.Key = bytes;
}
catch (Exception _ex) when (!_ex.Message.Contains("scheme is not supported."))
{
Logger.WarnMarkUp($"[grey]{_ex.Message.EscapeMarkup()} retryCount: {retryCount}[/]");
Thread.Sleep(1000);
if (retryCount-- > 0) goto getHttpKey;
throw;
}
}
}
catch (Exception ex)
{
Logger.Error(ResString.cmd_loadKeyFailed + ": " + ex.Message);
encryptInfo.Method = EncryptMethod.UNKNOWN;
}
if (parserConfig.CustomMethod == null) return encryptInfo;
// 处理自定义加密方式
encryptInfo.Method = parserConfig.CustomMethod.Value;
Logger.Warn("METHOD changed from {} to {}", method, encryptInfo.Method);
return encryptInfo;
}
/// <summary>
/// 预处理URL
/// </summary>
private string PreProcessUrl(string url, ParserConfig parserConfig)
{
foreach (var p in parserConfig.UrlProcessors)
{
if (p.CanProcess(ExtractorType.HLS, url, parserConfig))
{
url = p.Process(url, parserConfig);
}
}
return url;
}
}

11
src/N_m3u8DL-RE.Parser/Processor/KeyProcessor.cs Normal file
View File

@ -0,0 +1,11 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Parser.Config;
namespace N_m3u8DL_RE.Parser.Processor;
public abstract class KeyProcessor
{
public abstract bool CanProcess(ExtractorType extractorType, string keyLine, string m3u8Url, string m3u8Content, ParserConfig parserConfig);
public abstract EncryptInfo Process(string keyLine, string m3u8Url, string m3u8Content, ParserConfig parserConfig);
}

10
src/N_m3u8DL-RE.Parser/Processor/UrlProcessor.cs Normal file
View File

@ -0,0 +1,10 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Parser.Config;
namespace N_m3u8DL_RE.Parser.Processor;
public abstract class UrlProcessor
{
public abstract bool CanProcess(ExtractorType extractorType, string oriUrl, ParserConfig parserConfig);
public abstract string Process(string oriUrl, ParserConfig parserConfig);
}

144
src/N_m3u8DL-RE.Parser/StreamExtractor.cs Normal file
View File

@ -0,0 +1,144 @@
using System.Diagnostics.CodeAnalysis;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Parser.Constants;
using N_m3u8DL_RE.Parser.Extractor;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Common.Enum;
namespace N_m3u8DL_RE.Parser;
public class StreamExtractor
{
public ExtractorType ExtractorType => extractor.ExtractorType;
private IExtractor extractor;
private ParserConfig parserConfig = new();
private string rawText;
private static SemaphoreSlim semaphore = new(1, 1);
public Dictionary<string, string> RawFiles { get; set; } = new(); // 存储（文件名,文件内容）
public StreamExtractor(ParserConfig parserConfig)
{
this.parserConfig = parserConfig;
}
public async Task LoadSourceFromUrlAsync(string url)
{
Logger.Info(ResString.loadingUrl + url);
if (url.StartsWith("file:"))
{
var uri = new Uri(url);
this.rawText = await File.ReadAllTextAsync(uri.LocalPath);
parserConfig.OriginalUrl = parserConfig.Url = url;
}
else if (url.StartsWith("http"))
{
parserConfig.OriginalUrl = url;
(this.rawText, url) = await HTTPUtil.GetWebSourceAndNewUrlAsync(url, parserConfig.Headers);
parserConfig.Url = url;
}
else if (File.Exists(url))
{
url = Path.GetFullPath(url);
this.rawText = await File.ReadAllTextAsync(url);
parserConfig.OriginalUrl = parserConfig.Url = new Uri(url).AbsoluteUri;
}
this.rawText = rawText.Trim();
LoadSourceFromText(this.rawText);
}
[MemberNotNull(nameof(this.rawText), nameof(this.extractor))]
private void LoadSourceFromText(string rawText)
{
var rawType = "txt";
rawText = rawText.Trim();
this.rawText = rawText;
if (rawText.StartsWith(HLSTags.ext_m3u))
{
Logger.InfoMarkUp(ResString.matchHLS);
extractor = new HLSExtractor(parserConfig);
rawType = "m3u8";
}
else if (rawText.Contains("</MPD>") && rawText.Contains("<MPD"))
{
Logger.InfoMarkUp(ResString.matchDASH);
// extractor = new DASHExtractor(parserConfig);
extractor = new DASHExtractor2(parserConfig);
rawType = "mpd";
}
else if (rawText.Contains("</SmoothStreamingMedia>") && rawText.Contains("<SmoothStreamingMedia"))
{
Logger.InfoMarkUp(ResString.matchMSS);
// extractor = new DASHExtractor(parserConfig);
extractor = new MSSExtractor(parserConfig);
rawType = "ism";
}
else if (rawText == ResString.ReLiveTs)
{
Logger.InfoMarkUp(ResString.matchTS);
extractor = new LiveTSExtractor(parserConfig);
}
else
{
throw new NotSupportedException(ResString.notSupported);
}
RawFiles[$"raw.{rawType}"] = rawText;
}
/// <summary>
/// 开始解析流媒体信息
/// </summary>
/// <returns></returns>
public async Task<List<StreamSpec>> ExtractStreamsAsync()
{
try
{
await semaphore.WaitAsync();
Logger.Info(ResString.parsingStream);
return await extractor.ExtractStreamsAsync(rawText);
}
finally
{
semaphore.Release();
}
}
/// <summary>
/// 根据规格说明填充媒体播放列表信息
/// </summary>
/// <param name="streamSpecs"></param>
public async Task FetchPlayListAsync(List<StreamSpec> streamSpecs)
{
try
{
await semaphore.WaitAsync();
Logger.Info(ResString.parsingStream);
await extractor.FetchPlayListAsync(streamSpecs);
}
finally
{
semaphore.Release();
}
}
public async Task RefreshPlayListAsync(List<StreamSpec> streamSpecs)
{
try
{
await semaphore.WaitAsync();
await RetryUtil.WebRequestRetryAsync(async () =>
{
await extractor.RefreshPlayListAsync(streamSpecs);
return true;
}, retryDelayMilliseconds: 1000, maxRetries: 5);
}
finally
{
semaphore.Release();
}
}
}

114
src/N_m3u8DL-RE.Parser/Util/ParserUtil.cs Normal file
View File

@ -0,0 +1,114 @@
using N_m3u8DL_RE.Parser.Constants;
using System.Text.RegularExpressions;
namespace N_m3u8DL_RE.Parser.Util;
public static partial class ParserUtil
{
[GeneratedRegex(@"\$Number%([^$]+)d\$")]
private static partial Regex VarsNumberRegex();
/// <summary>
/// 从以下文本中获取参数
/// #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2149280,CODECS="mp4a.40.2,avc1.64001f",RESOLUTION=1280x720,NAME="720"
/// </summary>
/// <param name="line">等待被解析的一行文本</param>
/// <param name="key">留空则获取第一个英文冒号后的全部字符</param>
/// <returns></returns>
public static string GetAttribute(string line, string key = "")
{
line = line.Trim();
if (key == "")
return line[(line.IndexOf(':') + 1)..];
var index = -1;
var result = string.Empty;
if ((index = line.IndexOf(key + "=\"", StringComparison.Ordinal)) > -1)
{
var startIndex = index + (key + "=\"").Length;
var endIndex = startIndex + line[startIndex..].IndexOf('\"');
result = line[startIndex..endIndex];
}
else if ((index = line.IndexOf(key + "=", StringComparison.Ordinal)) > -1)
{
var startIndex = index + (key + "=").Length;
var endIndex = startIndex + line[startIndex..].IndexOf(',');
result = endIndex >= startIndex ? line[startIndex..endIndex] : line[startIndex..];
}
return result;
}
/// <summary>
/// 从如下文本中提取
/// <n>[@<o>]
/// </summary>
/// <param name="input"></param>
/// <returns>n(length) o(start)</returns>
public static (long, long?) GetRange(string input)
{
var t = input.Split('@');
return t.Length switch
{
<= 0 => (0, null),
1 => (Convert.ToInt64(t[0]), null),
2 => (Convert.ToInt64(t[0]), Convert.ToInt64(t[1])),
_ => (0, null)
};
}
/// <summary>
/// 从100-300这种字符串中获取StartRange, ExpectLength信息
/// </summary>
/// <param name="range"></param>
/// <returns>StartRange, ExpectLength</returns>
public static (long, long) ParseRange(string range)
{
var start = Convert.ToInt64(range.Split('-')[0]);
var end = Convert.ToInt64(range.Split('-')[1]);
return (start, end - start + 1);
}
/// <summary>
/// MPD SegmentTemplate替换
/// </summary>
/// <param name="text"></param>
/// <param name="keyValuePairs"></param>
/// <returns></returns>
public static string ReplaceVars(string text, Dictionary<string, object?> keyValuePairs)
{
foreach (var item in keyValuePairs)
if (text.Contains(item.Key))
text = text.Replace(item.Key, item.Value!.ToString());
// 处理特殊形式数字 如 $Number%05d$
var regex = VarsNumberRegex();
if (regex.IsMatch(text) && keyValuePairs.TryGetValue(DASHTags.TemplateNumber, out var keyValuePair))
{
foreach (Match m in regex.Matches(text))
{
text = text.Replace(m.Value, keyValuePair?.ToString()?.PadLeft(Convert.ToInt32(m.Groups[1].Value), '0'));
}
}
return text;
}
/// <summary>
/// 拼接Baseurl和RelativeUrl
/// </summary>
/// <param name="baseurl">Baseurl</param>
/// <param name="url">RelativeUrl</param>
/// <returns></returns>
public static string CombineURL(string baseurl, string url)
{
if (string.IsNullOrEmpty(baseurl))
return url;
var uri1 = new Uri(baseurl); // 这里直接传完整的URL即可
var uri2 = new Uri(uri1, url);
url = uri2.ToString();
return url;
}
}

38
src/N_m3u8DL-RE.sln Normal file
View File

@ -0,0 +1,38 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 17
VisualStudioVersion = 17.3.32505.426
MinimumVisualStudioVersion = 10.0.40219.1
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "N_m3u8DL-RE", "N_m3u8DL-RE\N_m3u8DL-RE.csproj", "{E6915BF9-8306-4F62-B357-23430F0D80B5}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "N_m3u8DL-RE.Common", "N_m3u8DL-RE.Common\N_m3u8DL-RE.Common.csproj", "{18D8BC30-512C-4A07-BD4C-E96DF5CF6341}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "N_m3u8DL-RE.Parser", "N_m3u8DL-RE.Parser\N_m3u8DL-RE.Parser.csproj", "{0DA02925-AF3A-4598-AF01-91AE5539FCA1}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Release|Any CPU = Release|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{E6915BF9-8306-4F62-B357-23430F0D80B5}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{E6915BF9-8306-4F62-B357-23430F0D80B5}.Debug|Any CPU.Build.0 = Debug|Any CPU
{E6915BF9-8306-4F62-B357-23430F0D80B5}.Release|Any CPU.ActiveCfg = Release|Any CPU
{E6915BF9-8306-4F62-B357-23430F0D80B5}.Release|Any CPU.Build.0 = Release|Any CPU
{18D8BC30-512C-4A07-BD4C-E96DF5CF6341}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{18D8BC30-512C-4A07-BD4C-E96DF5CF6341}.Debug|Any CPU.Build.0 = Debug|Any CPU
{18D8BC30-512C-4A07-BD4C-E96DF5CF6341}.Release|Any CPU.ActiveCfg = Release|Any CPU
{18D8BC30-512C-4A07-BD4C-E96DF5CF6341}.Release|Any CPU.Build.0 = Release|Any CPU
{0DA02925-AF3A-4598-AF01-91AE5539FCA1}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{0DA02925-AF3A-4598-AF01-91AE5539FCA1}.Debug|Any CPU.Build.0 = Debug|Any CPU
{0DA02925-AF3A-4598-AF01-91AE5539FCA1}.Release|Any CPU.ActiveCfg = Release|Any CPU
{0DA02925-AF3A-4598-AF01-91AE5539FCA1}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
RESX_NeutralResourcesLanguage = en-US
SolutionGuid = {87F963D4-EA06-413D-9372-C726711C32B5}
EndGlobalSection
EndGlobal

48
src/N_m3u8DL-RE/Column/DownloadSpeedColumn.cs Normal file
View File

@ -0,0 +1,48 @@
using N_m3u8DL_RE.Entity;
using Spectre.Console;
using Spectre.Console.Rendering;
using System.Collections.Concurrent;
using N_m3u8DL_RE.Common.Util;
namespace N_m3u8DL_RE.Column;
internal sealed class DownloadSpeedColumn : ProgressColumn
{
private long _stopSpeed = 0;
private ConcurrentDictionary<int, string> DateTimeStringDic = new();
protected override bool NoWrap => true;
private ConcurrentDictionary<int, SpeedContainer> SpeedContainerDic { get; set; }
public DownloadSpeedColumn(ConcurrentDictionary<int, SpeedContainer> SpeedContainerDic)
{
this.SpeedContainerDic = SpeedContainerDic;
}
public Style MyStyle { get; set; } = new Style(foreground: Color.Green);
public override IRenderable Render(RenderOptions options, ProgressTask task, TimeSpan deltaTime)
{
var taskId = task.Id;
var speedContainer = SpeedContainerDic[taskId];
var now = DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss");
var flag = task.IsFinished || !task.IsStarted;
// 单文件下载汇报进度
if (!flag && speedContainer is { SingleSegment: true, ResponseLength: not null })
{
task.MaxValue = (double)speedContainer.ResponseLength;
task.Value = speedContainer.RDownloaded;
}
// 一秒汇报一次即可
if (DateTimeStringDic.TryGetValue(taskId, out var oldTime) && oldTime != now && !flag)
{
speedContainer.NowSpeed = speedContainer.Downloaded;
// 速度为0，计数增加
if (speedContainer.Downloaded <= _stopSpeed) { speedContainer.AddLowSpeedCount(); }
else speedContainer.ResetLowSpeedCount();
speedContainer.Reset();
}
DateTimeStringDic[taskId] = now;
var style = flag ? Style.Plain : MyStyle;
return flag ? new Text("-", style).Centered() : new Text(GlobalUtil.FormatFileSize(speedContainer.NowSpeed) + "ps" + (speedContainer.LowSpeedCount > 0 ? $"({speedContainer.LowSpeedCount})" : ""), style).Centered();
}
}

43
src/N_m3u8DL-RE/Column/DownloadStatusColumn.cs Normal file
View File

@ -0,0 +1,43 @@
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Entity;
using Spectre.Console;
using Spectre.Console.Rendering;
using System.Collections.Concurrent;
namespace N_m3u8DL_RE.Column;
internal class DownloadStatusColumn : ProgressColumn
{
private ConcurrentDictionary<int, SpeedContainer> SpeedContainerDic { get; set; }
private ConcurrentDictionary<int, string> DateTimeStringDic = new();
private ConcurrentDictionary<int, string> SizeDic = new();
public Style MyStyle { get; set; } = new Style(foreground: Color.DarkCyan);
public Style FinishedStyle { get; set; } = new Style(foreground: Color.Green);
public DownloadStatusColumn(ConcurrentDictionary<int, SpeedContainer> speedContainerDic)
{
this.SpeedContainerDic = speedContainerDic;
}
public override IRenderable Render(RenderOptions options, ProgressTask task, TimeSpan deltaTime)
{
if (task.Value == 0) return new Text("-", MyStyle).RightJustified();
var now = DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss");
var speedContainer = SpeedContainerDic[task.Id];
var size = speedContainer.RDownloaded;
// 一秒汇报一次即可
if (DateTimeStringDic.TryGetValue(task.Id, out var oldTime) && oldTime != now)
{
var totalSize = speedContainer.SingleSegment ? (speedContainer.ResponseLength ?? 0) : (long)(size / (task.Value / task.MaxValue));
SizeDic[task.Id] = $"{GlobalUtil.FormatFileSize(size)}/{GlobalUtil.FormatFileSize(totalSize)}";
}
DateTimeStringDic[task.Id] = now;
SizeDic.TryGetValue(task.Id, out var sizeStr);
if (task.IsFinished) sizeStr = GlobalUtil.FormatFileSize(size);
return new Text(sizeStr ?? "-", MyStyle).RightJustified();
}
}

25
src/N_m3u8DL-RE/Column/MyPercentageColumn.cs Normal file
View File

@ -0,0 +1,25 @@
using Spectre.Console.Rendering;
using Spectre.Console;
namespace N_m3u8DL_RE.Column;
internal class MyPercentageColumn : ProgressColumn
{
/// <summary>
/// Gets or sets the style for a non-complete task.
/// </summary>
public Style Style { get; set; } = Style.Plain;
/// <summary>
/// Gets or sets the style for a completed task.
/// </summary>
public Style CompletedStyle { get; set; } = new Style(foreground: Color.Green);
/// <inheritdoc/>
public override IRenderable Render(RenderOptions options, ProgressTask task, TimeSpan deltaTime)
{
var percentage = task.Percentage;
var style = percentage == 100 ? CompletedStyle : Style ?? Style.Plain;
return new Text($"{task.Value}/{task.MaxValue} {percentage:F2}%", style).RightJustified();
}
}

30
src/N_m3u8DL-RE/Column/RecordingDurationColumn.cs Normal file
View File

@ -0,0 +1,30 @@
using N_m3u8DL_RE.Common.Util;
using Spectre.Console;
using Spectre.Console.Rendering;
using System.Collections.Concurrent;
namespace N_m3u8DL_RE.Column;
internal class RecordingDurationColumn : ProgressColumn
{
protected override bool NoWrap => true;
private ConcurrentDictionary<int, int> _recodingDurDic;
private ConcurrentDictionary<int, int>? _refreshedDurDic;
public Style GreyStyle { get; set; } = new Style(foreground: Color.Grey);
public Style MyStyle { get; set; } = new Style(foreground: Color.DarkGreen);
public RecordingDurationColumn(ConcurrentDictionary<int, int> recodingDurDic)
{
_recodingDurDic = recodingDurDic;
}
public RecordingDurationColumn(ConcurrentDictionary<int, int> recodingDurDic, ConcurrentDictionary<int, int> refreshedDurDic)
{
_recodingDurDic = recodingDurDic;
_refreshedDurDic = refreshedDurDic;
}
public override IRenderable Render(RenderOptions options, ProgressTask task, TimeSpan deltaTime)
{
if (_refreshedDurDic == null)
return new Text($"{GlobalUtil.FormatTime(_recodingDurDic[task.Id])}", MyStyle).LeftJustified();
return new Text($"{GlobalUtil.FormatTime(_recodingDurDic[task.Id])}/{GlobalUtil.FormatTime(_refreshedDurDic[task.Id])}", GreyStyle);
}
}

32
src/N_m3u8DL-RE/Column/RecordingSizeColumn.cs Normal file
View File

@ -0,0 +1,32 @@
using N_m3u8DL_RE.Common.Util;
using Spectre.Console;
using Spectre.Console.Rendering;
using System.Collections.Concurrent;
namespace N_m3u8DL_RE.Column;
internal class RecordingSizeColumn : ProgressColumn
{
protected override bool NoWrap => true;
private ConcurrentDictionary<int, double> RecodingSizeDic = new(); // 临时的大小 每秒刷新用
private ConcurrentDictionary<int, double> _recodingSizeDic;
private ConcurrentDictionary<int, string> DateTimeStringDic = new();
public Style MyStyle { get; set; } = new Style(foreground: Color.DarkCyan);
public RecordingSizeColumn(ConcurrentDictionary<int, double> recodingSizeDic)
{
_recodingSizeDic = recodingSizeDic;
}
public override IRenderable Render(RenderOptions options, ProgressTask task, TimeSpan deltaTime)
{
var now = DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss");
var taskId = task.Id;
// 一秒汇报一次即可
if (DateTimeStringDic.TryGetValue(taskId, out var oldTime) && oldTime != now)
{
RecodingSizeDic[task.Id] = _recodingSizeDic[task.Id];
}
DateTimeStringDic[taskId] = now;
var flag = RecodingSizeDic.TryGetValue(taskId, out var size);
return new Text(GlobalUtil.FormatFileSize(flag ? size : 0), MyStyle).LeftJustified();
}
}

17
src/N_m3u8DL-RE/Column/RecordingStatusColumn.cs Normal file
View File

@ -0,0 +1,17 @@
using Spectre.Console;
using Spectre.Console.Rendering;
namespace N_m3u8DL_RE.Column;
internal class RecordingStatusColumn : ProgressColumn
{
protected override bool NoWrap => true;
public Style MyStyle { get; set; } = new Style(foreground: Color.Default);
public Style FinishedStyle { get; set; } = new Style(foreground: Color.Yellow);
public override IRenderable Render(RenderOptions options, ProgressTask task, TimeSpan deltaTime)
{
if (task.IsFinished)
return new Text($"{task.Value}/{task.MaxValue} Waiting ", FinishedStyle).LeftJustified();
return new Text($"{task.Value}/{task.MaxValue} Recording", MyStyle).LeftJustified();
}
}

648
src/N_m3u8DL-RE/CommandLine/CommandInvoker.cs Normal file
View File

@ -0,0 +1,648 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Entity;
using N_m3u8DL_RE.Enum;
using N_m3u8DL_RE.Util;
using System.CommandLine;
using System.CommandLine.Binding;
using System.CommandLine.Builder;
using System.CommandLine.Parsing;
using System.Globalization;
using System.Net;
using System.Text.RegularExpressions;
namespace N_m3u8DL_RE.CommandLine;
internal static partial class CommandInvoker
{
public const string VERSION_INFO = "N_m3u8DL-RE (Beta version) 20241216";
[GeneratedRegex("((best|worst)\\d*|all)")]
private static partial Regex ForStrRegex();
[GeneratedRegex(@"(\d*)-(\d*)")]
private static partial Regex RangeRegex();
[GeneratedRegex(@"([\d\\.]+)(M|K)")]
private static partial Regex SpeedStrRegex();
private static readonly Argument<string> Input = new(name: "input", description: ResString.cmd_Input);
private static readonly Option<string?> TmpDir = new(["--tmp-dir"], description: ResString.cmd_tmpDir);
private static readonly Option<string?> SaveDir = new(["--save-dir"], description: ResString.cmd_saveDir);
private static readonly Option<string?> SaveName = new(["--save-name"], description: ResString.cmd_saveName, parseArgument: ParseSaveName);
private static readonly Option<string?> SavePattern = new(["--save-pattern"], description: ResString.cmd_savePattern, getDefaultValue: () => "<SaveName>_<Id>_<Codecs>_<Language>_<Ext>");
private static readonly Option<string?> UILanguage = new Option<string?>(["--ui-language"], description: ResString.cmd_uiLanguage).FromAmong("en-US", "zh-CN", "zh-TW");
private static readonly Option<string?> UrlProcessorArgs = new(["--urlprocessor-args"], description: ResString.cmd_urlProcessorArgs);
private static readonly Option<string[]?> Keys = new(["--key"], description: ResString.cmd_keys) { Arity = ArgumentArity.OneOrMore, AllowMultipleArgumentsPerToken = false };
private static readonly Option<string> KeyTextFile = new(["--key-text-file"], description: ResString.cmd_keyText);
private static readonly Option<Dictionary<string, string>> Headers = new(["-H", "--header"], description: ResString.cmd_header, parseArgument: ParseHeaders) { Arity = ArgumentArity.OneOrMore, AllowMultipleArgumentsPerToken = false };
private static readonly Option<LogLevel> LogLevel = new(name: "--log-level", description: ResString.cmd_logLevel, getDefaultValue: () => Common.Log.LogLevel.INFO);
private static readonly Option<SubtitleFormat> SubtitleFormat = new(name: "--sub-format", description: ResString.cmd_subFormat, getDefaultValue: () => Enum.SubtitleFormat.SRT);
private static readonly Option<bool> DisableUpdateCheck = new(["--disable-update-check"], description: ResString.cmd_disableUpdateCheck, getDefaultValue: () => false);
private static readonly Option<bool> AutoSelect = new(["--auto-select"], description: ResString.cmd_autoSelect, getDefaultValue: () => false);
private static readonly Option<bool> SubOnly = new(["--sub-only"], description: ResString.cmd_subOnly, getDefaultValue: () => false);
private static readonly Option<int> ThreadCount = new(["--thread-count"], description: ResString.cmd_threadCount, getDefaultValue: () => Environment.ProcessorCount) { ArgumentHelpName = "number" };
private static readonly Option<int> DownloadRetryCount = new(["--download-retry-count"], description: ResString.cmd_downloadRetryCount, getDefaultValue: () => 3) { ArgumentHelpName = "number" };
private static readonly Option<double> HttpRequestTimeout = new(["--http-request-timeout"], description: ResString.cmd_httpRequestTimeout, getDefaultValue: () => 100) { ArgumentHelpName = "seconds" };
private static readonly Option<bool> SkipMerge = new(["--skip-merge"], description: ResString.cmd_skipMerge, getDefaultValue: () => false);
private static readonly Option<bool> SkipDownload = new(["--skip-download"], description: ResString.cmd_skipDownload, getDefaultValue: () => false);
private static readonly Option<bool> NoDateInfo = new(["--no-date-info"], description: ResString.cmd_noDateInfo, getDefaultValue: () => false);
private static readonly Option<bool> BinaryMerge = new(["--binary-merge"], description: ResString.cmd_binaryMerge, getDefaultValue: () => false);
private static readonly Option<bool> UseFFmpegConcatDemuxer = new(["--use-ffmpeg-concat-demuxer"], description: ResString.cmd_useFFmpegConcatDemuxer, getDefaultValue: () => false);
private static readonly Option<bool> DelAfterDone = new(["--del-after-done"], description: ResString.cmd_delAfterDone, getDefaultValue: () => true);
private static readonly Option<bool> AutoSubtitleFix = new(["--auto-subtitle-fix"], description: ResString.cmd_subtitleFix, getDefaultValue: () => true);
private static readonly Option<bool> CheckSegmentsCount = new(["--check-segments-count"], description: ResString.cmd_checkSegmentsCount, getDefaultValue: () => true);
private static readonly Option<bool> WriteMetaJson = new(["--write-meta-json"], description: ResString.cmd_writeMetaJson, getDefaultValue: () => true);
private static readonly Option<bool> AppendUrlParams = new(["--append-url-params"], description: ResString.cmd_appendUrlParams, getDefaultValue: () => false);
private static readonly Option<bool> MP4RealTimeDecryption = new (["--mp4-real-time-decryption"], description: ResString.cmd_MP4RealTimeDecryption, getDefaultValue: () => false);
private static readonly Option<bool> UseShakaPackager = new (["--use-shaka-packager"], description: ResString.cmd_useShakaPackager, getDefaultValue: () => false) { IsHidden = true };
private static readonly Option<DecryptEngine> DecryptionEngine = new (["--decryption-engine"], description: ResString.cmd_decryptionEngine, getDefaultValue: () => DecryptEngine.MP4DECRYPT);
private static readonly Option<bool> ForceAnsiConsole = new(["--force-ansi-console"], description: ResString.cmd_forceAnsiConsole);
private static readonly Option<bool> NoAnsiColor = new(["--no-ansi-color"], description: ResString.cmd_noAnsiColor);
private static readonly Option<string?> DecryptionBinaryPath = new(["--decryption-binary-path"], description: ResString.cmd_decryptionBinaryPath) { ArgumentHelpName = "PATH" };
private static readonly Option<string?> FFmpegBinaryPath = new(["--ffmpeg-binary-path"], description: ResString.cmd_ffmpegBinaryPath) { ArgumentHelpName = "PATH" };
private static readonly Option<string?> BaseUrl = new(["--base-url"], description: ResString.cmd_baseUrl);
private static readonly Option<bool> ConcurrentDownload = new(["-mt", "--concurrent-download"], description: ResString.cmd_concurrentDownload, getDefaultValue: () => false);
private static readonly Option<bool> NoLog = new(["--no-log"], description: ResString.cmd_noLog, getDefaultValue: () => false);
private static readonly Option<bool> AllowHlsMultiExtMap = new(["--allow-hls-multi-ext-map"], description: ResString.cmd_allowHlsMultiExtMap, getDefaultValue: () => false);
private static readonly Option<string[]?> AdKeywords = new(["--ad-keyword"], description: ResString.cmd_adKeyword) { ArgumentHelpName = "REG" };
private static readonly Option<long?> MaxSpeed = new(["-R", "--max-speed"], description: ResString.cmd_maxSpeed, parseArgument: ParseSpeedLimit) { ArgumentHelpName = "SPEED" };
// 代理选项
private static readonly Option<bool> UseSystemProxy = new(["--use-system-proxy"], description: ResString.cmd_useSystemProxy, getDefaultValue: () => true);
private static readonly Option<WebProxy?> CustomProxy = new(["--custom-proxy"], description: ResString.cmd_customProxy, parseArgument: ParseProxy) { ArgumentHelpName = "URL" };
// 只下载部分分片
private static readonly Option<CustomRange?> CustomRange = new(["--custom-range"], description: ResString.cmd_customRange, parseArgument: ParseCustomRange) { ArgumentHelpName = "RANGE" };
// morehelp
private static readonly Option<string?> MoreHelp = new(["--morehelp"], description: ResString.cmd_moreHelp) { ArgumentHelpName = "OPTION" };
// 自定义KEY等
private static readonly Option<EncryptMethod?> CustomHLSMethod = new(name: "--custom-hls-method", description: ResString.cmd_customHLSMethod) { ArgumentHelpName = "METHOD" };
private static readonly Option<byte[]?> CustomHLSKey = new(name: "--custom-hls-key", description: ResString.cmd_customHLSKey, parseArgument: ParseHLSCustomKey) { ArgumentHelpName = "FILE|HEX|BASE64" };
private static readonly Option<byte[]?> CustomHLSIv = new(name: "--custom-hls-iv", description: ResString.cmd_customHLSIv, parseArgument: ParseHLSCustomKey) { ArgumentHelpName = "FILE|HEX|BASE64" };
// 任务开始时间
private static readonly Option<DateTime?> TaskStartAt = new(["--task-start-at"], description: ResString.cmd_taskStartAt, parseArgument: ParseStartTime) { ArgumentHelpName = "yyyyMMddHHmmss" };
// 直播相关
private static readonly Option<bool> LivePerformAsVod = new(["--live-perform-as-vod"], description: ResString.cmd_livePerformAsVod, getDefaultValue: () => false);
private static readonly Option<bool> LiveRealTimeMerge = new(["--live-real-time-merge"], description: ResString.cmd_liveRealTimeMerge, getDefaultValue: () => false);
private static readonly Option<bool> LiveKeepSegments = new(["--live-keep-segments"], description: ResString.cmd_liveKeepSegments, getDefaultValue: () => true);
private static readonly Option<bool> LivePipeMux = new(["--live-pipe-mux"], description: ResString.cmd_livePipeMux, getDefaultValue: () => false);
private static readonly Option<TimeSpan?> LiveRecordLimit = new(["--live-record-limit"], description: ResString.cmd_liveRecordLimit, parseArgument: ParseLiveLimit) { ArgumentHelpName = "HH:mm:ss" };
private static readonly Option<int?> LiveWaitTime = new(["--live-wait-time"], description: ResString.cmd_liveWaitTime) { ArgumentHelpName = "SEC" };
private static readonly Option<int> LiveTakeCount = new(["--live-take-count"], description: ResString.cmd_liveTakeCount, getDefaultValue: () => 16) { ArgumentHelpName = "NUM" };
private static readonly Option<bool> LiveFixVttByAudio = new(["--live-fix-vtt-by-audio"], description: ResString.cmd_liveFixVttByAudio, getDefaultValue: () => false);
// 复杂命令行如下
private static readonly Option<MuxOptions?> MuxAfterDone = new(["-M", "--mux-after-done"], description: ResString.cmd_muxAfterDone, parseArgument: ParseMuxAfterDone) { ArgumentHelpName = "OPTIONS" };
private static readonly Option<List<OutputFile>> MuxImports = new("--mux-import", description: ResString.cmd_muxImport, parseArgument: ParseImports) { Arity = ArgumentArity.OneOrMore, AllowMultipleArgumentsPerToken = false, ArgumentHelpName = "OPTIONS" };
private static readonly Option<StreamFilter?> VideoFilter = new(["-sv", "--select-video"], description: ResString.cmd_selectVideo, parseArgument: ParseStreamFilter) { ArgumentHelpName = "OPTIONS" };
private static readonly Option<StreamFilter?> AudioFilter = new(["-sa", "--select-audio"], description: ResString.cmd_selectAudio, parseArgument: ParseStreamFilter) { ArgumentHelpName = "OPTIONS" };
private static readonly Option<StreamFilter?> SubtitleFilter = new(["-ss", "--select-subtitle"], description: ResString.cmd_selectSubtitle, parseArgument: ParseStreamFilter) { ArgumentHelpName = "OPTIONS" };
private static readonly Option<StreamFilter?> DropVideoFilter = new(["-dv", "--drop-video"], description: ResString.cmd_dropVideo, parseArgument: ParseStreamFilter) { ArgumentHelpName = "OPTIONS" };
private static readonly Option<StreamFilter?> DropAudioFilter = new(["-da", "--drop-audio"], description: ResString.cmd_dropAudio, parseArgument: ParseStreamFilter) { ArgumentHelpName = "OPTIONS" };
private static readonly Option<StreamFilter?> DropSubtitleFilter = new(["-ds", "--drop-subtitle"], description: ResString.cmd_dropSubtitle, parseArgument: ParseStreamFilter) { ArgumentHelpName = "OPTIONS" };
/// <summary>
/// 解析下载速度限制
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
private static long? ParseSpeedLimit(ArgumentResult result)
{
var input = result.Tokens[0].Value.ToUpper();
try
{
var reg = SpeedStrRegex();
if (!reg.IsMatch(input)) throw new ArgumentException($"Invalid Speed Limit: {input}");
var number = double.Parse(reg.Match(input).Groups[1].Value);
if (reg.Match(input).Groups[2].Value == "M")
return (long)(number * 1024 * 1024);
return (long)(number * 1024);
}
catch (Exception)
{
result.ErrorMessage = "error in parse SpeedLimit: " + input;
return null;
}
}
/// <summary>
/// 解析用户定义的下载范围
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
/// <exception cref="ArgumentException"></exception>
private static CustomRange? ParseCustomRange(ArgumentResult result)
{
var input = result.Tokens[0].Value;
// 支持的种类 0-100; 01:00:00-02:30:00; -300; 300-; 05:00-; -03:00;
try
{
if (string.IsNullOrEmpty(input))
return null;
var arr = input.Split('-');
if (arr.Length != 2)
throw new ArgumentException("Bad format!");
if (input.Contains(':'))
{
return new CustomRange()
{
InputStr = input,
StartSec = arr[0] == "" ? 0 : OtherUtil.ParseDur(arr[0]).TotalSeconds,
EndSec = arr[1] == "" ? double.MaxValue : OtherUtil.ParseDur(arr[1]).TotalSeconds,
};
}
if (RangeRegex().IsMatch(input))
{
var left = RangeRegex().Match(input).Groups[1].Value;
var right = RangeRegex().Match(input).Groups[2].Value;
return new CustomRange()
{
InputStr = input,
StartSegIndex = left == "" ? 0 : long.Parse(left),
EndSegIndex = right == "" ? long.MaxValue : long.Parse(right),
};
}
throw new ArgumentException("Bad format!");
}
catch (Exception ex)
{
result.ErrorMessage = $"error in parse CustomRange: " + ex.Message;
return null;
}
}
/// <summary>
/// 解析用户代理
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
/// <exception cref="ArgumentException"></exception>
private static WebProxy? ParseProxy(ArgumentResult result)
{
var input = result.Tokens[0].Value;
try
{
if (string.IsNullOrEmpty(input))
return null;
var uri = new Uri(input);
var proxy = new WebProxy(uri, true);
if (!string.IsNullOrEmpty(uri.UserInfo))
{
var infos = uri.UserInfo.Split(':');
proxy.Credentials = new NetworkCredential(infos.First(), infos.Last());
}
return proxy;
}
catch (Exception ex)
{
result.ErrorMessage = $"error in parse proxy: " + ex.Message;
return null;
}
}
/// <summary>
/// 解析自定义KEY
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
private static byte[]? ParseHLSCustomKey(ArgumentResult result)
{
var input = result.Tokens[0].Value;
try
{
if (string.IsNullOrEmpty(input))
return null;
if (File.Exists(input))
return File.ReadAllBytes(input);
if (HexUtil.TryParseHexString(input, out byte[]? bytes))
return bytes;
return Convert.FromBase64String(input);
}
catch (Exception)
{
result.ErrorMessage = "error in parse hls custom key: " + input;
return null;
}
}
/// <summary>
/// 解析录制直播时长限制
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
private static TimeSpan? ParseLiveLimit(ArgumentResult result)
{
var input = result.Tokens[0].Value;
try
{
return OtherUtil.ParseDur(input);
}
catch (Exception)
{
result.ErrorMessage = "error in parse LiveRecordLimit: " + input;
return null;
}
}
/// <summary>
/// 解析任务开始时间
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
private static DateTime? ParseStartTime(ArgumentResult result)
{
var input = result.Tokens[0].Value;
try
{
CultureInfo provider = CultureInfo.InvariantCulture;
return DateTime.ParseExact(input, "yyyyMMddHHmmss", provider);
}
catch (Exception)
{
result.ErrorMessage = "error in parse TaskStartTime: " + input;
return null;
}
}
private static string? ParseSaveName(ArgumentResult result)
{
var input = result.Tokens[0].Value;
var newName = OtherUtil.GetValidFileName(input);
if (string.IsNullOrEmpty(newName))
{
result.ErrorMessage = "Invalid save name!";
return null;
}
return newName;
}
/// <summary>
/// 流过滤器
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
private static StreamFilter? ParseStreamFilter(ArgumentResult result)
{
var streamFilter = new StreamFilter();
var input = result.Tokens[0].Value;
var p = new ComplexParamParser(input);
// 目标范围
var forStr = "";
if (input == ForStrRegex().Match(input).Value)
{
forStr = input;
}
else
{
forStr = p.GetValue("for") ?? "best";
if (forStr != ForStrRegex().Match(forStr).Value)
{
result.ErrorMessage = $"for={forStr} not valid";
return null;
}
}
streamFilter.For = forStr;
var id = p.GetValue("id");
if (!string.IsNullOrEmpty(id))
streamFilter.GroupIdReg = new Regex(id);
var lang = p.GetValue("lang");
if (!string.IsNullOrEmpty(lang))
streamFilter.LanguageReg = new Regex(lang);
var name = p.GetValue("name");
if (!string.IsNullOrEmpty(name))
streamFilter.NameReg = new Regex(name);
var codecs = p.GetValue("codecs");
if (!string.IsNullOrEmpty(codecs))
streamFilter.CodecsReg = new Regex(codecs);
var res = p.GetValue("res");
if (!string.IsNullOrEmpty(res))
streamFilter.ResolutionReg = new Regex(res);
var frame = p.GetValue("frame");
if (!string.IsNullOrEmpty(frame))
streamFilter.FrameRateReg = new Regex(frame);
var channel = p.GetValue("channel");
if (!string.IsNullOrEmpty(channel))
streamFilter.ChannelsReg = new Regex(channel);
var range = p.GetValue("range");
if (!string.IsNullOrEmpty(range))
streamFilter.VideoRangeReg = new Regex(range);
var url = p.GetValue("url");
if (!string.IsNullOrEmpty(url))
streamFilter.UrlReg = new Regex(url);
var segsMin = p.GetValue("segsMin");
if (!string.IsNullOrEmpty(segsMin))
streamFilter.SegmentsMinCount = long.Parse(segsMin);
var segsMax = p.GetValue("segsMax");
if (!string.IsNullOrEmpty(segsMax))
streamFilter.SegmentsMaxCount = long.Parse(segsMax);
var plistDurMin = p.GetValue("plistDurMin");
if (!string.IsNullOrEmpty(plistDurMin))
streamFilter.PlaylistMinDur = OtherUtil.ParseSeconds(plistDurMin);
var plistDurMax = p.GetValue("plistDurMax");
if (!string.IsNullOrEmpty(plistDurMax))
streamFilter.PlaylistMaxDur = OtherUtil.ParseSeconds(plistDurMax);
var bwMin = p.GetValue("bwMin");
if (!string.IsNullOrEmpty(bwMin))
streamFilter.BandwidthMin = int.Parse(bwMin) * 1000;
var bwMax = p.GetValue("bwMax");
if (!string.IsNullOrEmpty(bwMax))
streamFilter.BandwidthMax = int.Parse(bwMax) * 1000;
var role = p.GetValue("role");
if (System.Enum.TryParse(role, true, out RoleType roleType))
streamFilter.Role = roleType;
return streamFilter;
}
/// <summary>
/// 分割Header
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
private static Dictionary<string, string> ParseHeaders(ArgumentResult result)
{
var array = result.Tokens.Select(t => t.Value).ToArray();
return OtherUtil.SplitHeaderArrayToDic(array);
}
/// <summary>
/// 解析混流引入的外部文件
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
private static List<OutputFile> ParseImports(ArgumentResult result)
{
var imports = new List<OutputFile>();
foreach (var item in result.Tokens)
{
var p = new ComplexParamParser(item.Value);
var path = p.GetValue("path") ?? item.Value; // 若未获取到，直接整个字符串作为path
var lang = p.GetValue("lang");
var name = p.GetValue("name");
if (string.IsNullOrEmpty(path) || !File.Exists(path))
{
result.ErrorMessage = "path empty or file not exists!";
return imports;
}
imports.Add(new OutputFile()
{
Index = 999,
FilePath = path,
LangCode = lang,
Description = name
});
}
return imports;
}
/// <summary>
/// 解析混流选项
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
private static MuxOptions? ParseMuxAfterDone(ArgumentResult result)
{
var v = result.Tokens[0].Value;
var p = new ComplexParamParser(v);
// 混流格式
var format = p.GetValue("format") ?? v.Split(':')[0]; // 若未获取到，直接:前的字符串作为format解析
var parseResult = System.Enum.TryParse(format.ToUpperInvariant(), out MuxFormat muxFormat);
if (!parseResult)
{
result.ErrorMessage = $"format={format} not valid";
return null;
}
// 混流器
var muxer = p.GetValue("muxer") ?? "ffmpeg";
if (muxer != "ffmpeg" && muxer != "mkvmerge")
{
result.ErrorMessage = $"muxer={muxer} not valid";
return null;
}
// 混流器路径
var bin_path = p.GetValue("bin_path") ?? "auto";
if (string.IsNullOrEmpty(bin_path))
{
result.ErrorMessage = $"bin_path={bin_path} not valid";
return null;
}
// 是否删除
var keep = p.GetValue("keep") ?? "false";
if (keep != "true" && keep != "false")
{
result.ErrorMessage = $"keep={keep} not valid";
return null;
}
// 是否忽略字幕
var skipSub = p.GetValue("skip_sub") ?? "false";
if (skipSub != "true" && skipSub != "false")
{
result.ErrorMessage = $"skip_sub={keep} not valid";
return null;
}
// 冲突检测
if (muxer == "mkvmerge" && format == "mp4")
{
result.ErrorMessage = $"mkvmerge can not do mp4";
return null;
}
return new MuxOptions()
{
UseMkvmerge = muxer == "mkvmerge",
MuxFormat = muxFormat,
KeepFiles = keep == "true",
SkipSubtitle = skipSub == "true",
BinPath = bin_path == "auto" ? null : bin_path
};
}
class MyOptionBinder : BinderBase<MyOption>
{
protected override MyOption GetBoundValue(BindingContext bindingContext)
{
var option = new MyOption
{
Input = bindingContext.ParseResult.GetValueForArgument(Input),
ForceAnsiConsole = bindingContext.ParseResult.GetValueForOption(ForceAnsiConsole),
NoAnsiColor = bindingContext.ParseResult.GetValueForOption(NoAnsiColor),
LogLevel = bindingContext.ParseResult.GetValueForOption(LogLevel),
AutoSelect = bindingContext.ParseResult.GetValueForOption(AutoSelect),
DisableUpdateCheck = bindingContext.ParseResult.GetValueForOption(DisableUpdateCheck),
SkipMerge = bindingContext.ParseResult.GetValueForOption(SkipMerge),
BinaryMerge = bindingContext.ParseResult.GetValueForOption(BinaryMerge),
UseFFmpegConcatDemuxer = bindingContext.ParseResult.GetValueForOption(UseFFmpegConcatDemuxer),
DelAfterDone = bindingContext.ParseResult.GetValueForOption(DelAfterDone),
AutoSubtitleFix = bindingContext.ParseResult.GetValueForOption(AutoSubtitleFix),
CheckSegmentsCount = bindingContext.ParseResult.GetValueForOption(CheckSegmentsCount),
SubtitleFormat = bindingContext.ParseResult.GetValueForOption(SubtitleFormat),
SubOnly = bindingContext.ParseResult.GetValueForOption(SubOnly),
TmpDir = bindingContext.ParseResult.GetValueForOption(TmpDir),
SaveDir = bindingContext.ParseResult.GetValueForOption(SaveDir),
SaveName = bindingContext.ParseResult.GetValueForOption(SaveName),
ThreadCount = bindingContext.ParseResult.GetValueForOption(ThreadCount),
UILanguage = bindingContext.ParseResult.GetValueForOption(UILanguage),
SkipDownload = bindingContext.ParseResult.GetValueForOption(SkipDownload),
WriteMetaJson = bindingContext.ParseResult.GetValueForOption(WriteMetaJson),
AppendUrlParams = bindingContext.ParseResult.GetValueForOption(AppendUrlParams),
SavePattern = bindingContext.ParseResult.GetValueForOption(SavePattern),
Keys = bindingContext.ParseResult.GetValueForOption(Keys),
UrlProcessorArgs = bindingContext.ParseResult.GetValueForOption(UrlProcessorArgs),
MP4RealTimeDecryption = bindingContext.ParseResult.GetValueForOption(MP4RealTimeDecryption),
UseShakaPackager = bindingContext.ParseResult.GetValueForOption(UseShakaPackager),
DecryptionEngine = bindingContext.ParseResult.GetValueForOption(DecryptionEngine),
DecryptionBinaryPath = bindingContext.ParseResult.GetValueForOption(DecryptionBinaryPath),
FFmpegBinaryPath = bindingContext.ParseResult.GetValueForOption(FFmpegBinaryPath),
KeyTextFile = bindingContext.ParseResult.GetValueForOption(KeyTextFile),
DownloadRetryCount = bindingContext.ParseResult.GetValueForOption(DownloadRetryCount),
HttpRequestTimeout = bindingContext.ParseResult.GetValueForOption(HttpRequestTimeout),
BaseUrl = bindingContext.ParseResult.GetValueForOption(BaseUrl),
MuxImports = bindingContext.ParseResult.GetValueForOption(MuxImports),
ConcurrentDownload = bindingContext.ParseResult.GetValueForOption(ConcurrentDownload),
VideoFilter = bindingContext.ParseResult.GetValueForOption(VideoFilter),
AudioFilter = bindingContext.ParseResult.GetValueForOption(AudioFilter),
SubtitleFilter = bindingContext.ParseResult.GetValueForOption(SubtitleFilter),
DropVideoFilter = bindingContext.ParseResult.GetValueForOption(DropVideoFilter),
DropAudioFilter = bindingContext.ParseResult.GetValueForOption(DropAudioFilter),
DropSubtitleFilter = bindingContext.ParseResult.GetValueForOption(DropSubtitleFilter),
LiveRealTimeMerge = bindingContext.ParseResult.GetValueForOption(LiveRealTimeMerge),
LiveKeepSegments = bindingContext.ParseResult.GetValueForOption(LiveKeepSegments),
LiveRecordLimit = bindingContext.ParseResult.GetValueForOption(LiveRecordLimit),
TaskStartAt = bindingContext.ParseResult.GetValueForOption(TaskStartAt),
LivePerformAsVod = bindingContext.ParseResult.GetValueForOption(LivePerformAsVod),
LivePipeMux = bindingContext.ParseResult.GetValueForOption(LivePipeMux),
LiveFixVttByAudio = bindingContext.ParseResult.GetValueForOption(LiveFixVttByAudio),
UseSystemProxy = bindingContext.ParseResult.GetValueForOption(UseSystemProxy),
CustomProxy = bindingContext.ParseResult.GetValueForOption(CustomProxy),
CustomRange = bindingContext.ParseResult.GetValueForOption(CustomRange),
LiveWaitTime = bindingContext.ParseResult.GetValueForOption(LiveWaitTime),
LiveTakeCount = bindingContext.ParseResult.GetValueForOption(LiveTakeCount),
NoDateInfo = bindingContext.ParseResult.GetValueForOption(NoDateInfo),
NoLog = bindingContext.ParseResult.GetValueForOption(NoLog),
AllowHlsMultiExtMap = bindingContext.ParseResult.GetValueForOption(AllowHlsMultiExtMap),
AdKeywords = bindingContext.ParseResult.GetValueForOption(AdKeywords),
MaxSpeed = bindingContext.ParseResult.GetValueForOption(MaxSpeed),
};
if (bindingContext.ParseResult.HasOption(CustomHLSMethod)) option.CustomHLSMethod = bindingContext.ParseResult.GetValueForOption(CustomHLSMethod);
if (bindingContext.ParseResult.HasOption(CustomHLSKey)) option.CustomHLSKey = bindingContext.ParseResult.GetValueForOption(CustomHLSKey);
if (bindingContext.ParseResult.HasOption(CustomHLSIv)) option.CustomHLSIv = bindingContext.ParseResult.GetValueForOption(CustomHLSIv);
var parsedHeaders = bindingContext.ParseResult.GetValueForOption(Headers);
if (parsedHeaders != null)
option.Headers = parsedHeaders;
// 以用户选择语言为准优先
if (option.UILanguage != null)
{
CultureInfo.DefaultThreadCurrentCulture = new CultureInfo(option.UILanguage);
Thread.CurrentThread.CurrentUICulture = CultureInfo.GetCultureInfo(option.UILanguage);
Thread.CurrentThread.CurrentCulture = CultureInfo.GetCultureInfo(option.UILanguage);
}
// 混流设置
var muxAfterDoneValue = bindingContext.ParseResult.GetValueForOption(MuxAfterDone);
if (muxAfterDoneValue == null) return option;
option.MuxAfterDone = true;
option.MuxOptions = muxAfterDoneValue;
if (muxAfterDoneValue.UseMkvmerge) option.MkvmergeBinaryPath = muxAfterDoneValue.BinPath;
else option.FFmpegBinaryPath ??= muxAfterDoneValue.BinPath;
return option;
}
}
public static async Task<int> InvokeArgs(string[] args, Func<MyOption, Task> action)
{
var argList = new List<string>(args);
var index = -1;
if ((index = argList.IndexOf("--morehelp")) >= 0 && argList.Count > index + 1)
{
var option = argList[index + 1];
var msg = option switch
{
"mux-after-done" => ResString.cmd_muxAfterDone_more,
"mux-import" => ResString.cmd_muxImport_more,
"select-video" => ResString.cmd_selectVideo_more,
"select-audio" => ResString.cmd_selectAudio_more,
"select-subtitle" => ResString.cmd_selectSubtitle_more,
"custom-range" => ResString.cmd_custom_range,
_ => $"Option=\"{option}\" not found"
};
Console.WriteLine($"More Help:\r\n\r\n --{option}\r\n\r\n" + msg);
Environment.Exit(0);
}
var rootCommand = new RootCommand(VERSION_INFO)
{
Input, TmpDir, SaveDir, SaveName, BaseUrl, ThreadCount, DownloadRetryCount, HttpRequestTimeout, ForceAnsiConsole, NoAnsiColor,AutoSelect, SkipMerge, SkipDownload, CheckSegmentsCount,
BinaryMerge, UseFFmpegConcatDemuxer, DelAfterDone, NoDateInfo, NoLog, WriteMetaJson, AppendUrlParams, ConcurrentDownload, Headers, /**SavePattern,**/ SubOnly, SubtitleFormat, AutoSubtitleFix,
FFmpegBinaryPath,
LogLevel, UILanguage, UrlProcessorArgs, Keys, KeyTextFile, DecryptionEngine, DecryptionBinaryPath, UseShakaPackager, MP4RealTimeDecryption,
MaxSpeed,
MuxAfterDone,
CustomHLSMethod, CustomHLSKey, CustomHLSIv, UseSystemProxy, CustomProxy, CustomRange, TaskStartAt,
LivePerformAsVod, LiveRealTimeMerge, LiveKeepSegments, LivePipeMux, LiveFixVttByAudio, LiveRecordLimit, LiveWaitTime, LiveTakeCount,
MuxImports, VideoFilter, AudioFilter, SubtitleFilter, DropVideoFilter, DropAudioFilter, DropSubtitleFilter, AdKeywords, DisableUpdateCheck, AllowHlsMultiExtMap, MoreHelp
};
rootCommand.TreatUnmatchedTokensAsErrors = true;
rootCommand.SetHandler(async myOption => await action(myOption), new MyOptionBinder());
var parser = new CommandLineBuilder(rootCommand)
.UseDefaults()
.EnablePosixBundling(false)
.UseExceptionHandler((ex, context) =>
{
try { Console.CursorVisible = true; } catch { }
string msg = Logger.LogLevel == Common.Log.LogLevel.DEBUG ? ex.ToString() : ex.Message;
#if DEBUG
msg = ex.ToString();
#endif
Logger.Error(msg);
Thread.Sleep(3000);
Environment.Exit(1);
}, 1)
.Build();
return await parser.InvokeAsync(args);
}
}

56
src/N_m3u8DL-RE/CommandLine/ComplexParamParser.cs Normal file
View File

@ -0,0 +1,56 @@
using System.Text;
namespace N_m3u8DL_RE.CommandLine;
internal class ComplexParamParser
{
private readonly string _arg;
public ComplexParamParser(string arg)
{
_arg = arg;
}
public string? GetValue(string key)
{
if (string.IsNullOrEmpty(key) || string.IsNullOrEmpty(_arg)) return null;
try
{
var index = _arg.IndexOf(key + "=", StringComparison.Ordinal);
if (index == -1) return (_arg.Contains(key) && _arg.EndsWith(key)) ? "true" : null;
var chars = _arg[(index + key.Length + 1)..].ToCharArray();
var result = new StringBuilder();
char last = '\0';
for (int i = 0; i < chars.Length; i++)
{
if (chars[i] == ':')
{
if (last == '\\')
{
result.Replace("\\", "");
last = chars[i];
result.Append(chars[i]);
}
else break;
}
else
{
last = chars[i];
result.Append(chars[i]);
}
}
var resultStr = result.ToString().Trim().Trim('\"').Trim('\'');
// 不应该有引号出现
if (resultStr.Contains('\"') || resultStr.Contains('\'')) throw new Exception();
return resultStr;
}
catch (Exception)
{
throw new ArgumentException($"Parse Argument [{key}] failed!");
}
}
}

274
src/N_m3u8DL-RE/CommandLine/MyOption.cs Normal file
View File

@ -0,0 +1,274 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Entity;
using N_m3u8DL_RE.Enum;
using System.Net;
namespace N_m3u8DL_RE.CommandLine;
internal class MyOption
{
/// <summary>
/// See: <see cref="CommandInvoker.Input"/>.
/// </summary>
public string Input { get; set; } = default!;
/// <summary>
/// See: <see cref="CommandInvoker.Headers"/>.
/// </summary>
public Dictionary<string, string> Headers { get; set; } = new Dictionary<string, string>();
/// <summary>
/// See: <see cref="CommandInvoker.AdKeywords"/>.
/// </summary>
public string[]? AdKeywords { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.MaxSpeed"/>.
/// </summary>
public long? MaxSpeed { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.Keys"/>.
/// </summary>
public string[]? Keys { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.BaseUrl"/>.
/// </summary>
public string? BaseUrl { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.KeyTextFile"/>.
/// </summary>
public string? KeyTextFile { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.UrlProcessorArgs"/>.
/// </summary>
public string? UrlProcessorArgs { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.LogLevel"/>.
/// </summary>
public LogLevel LogLevel { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.NoDateInfo"/>.
/// </summary>
public bool NoDateInfo { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.NoLog"/>.
/// </summary>
public bool NoLog { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.AllowHlsMultiExtMap"/>.
/// </summary>
public bool AllowHlsMultiExtMap { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.AutoSelect"/>.
/// </summary>
public bool AutoSelect { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.DisableUpdateCheck"/>.
/// </summary>
public bool DisableUpdateCheck { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.SubOnly"/>.
/// </summary>
public bool SubOnly { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.ThreadCount"/>.
/// </summary>
public int ThreadCount { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.DownloadRetryCount"/>.
/// </summary>
public int DownloadRetryCount { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.HttpRequestTimeout"/>.
/// </summary>
public double HttpRequestTimeout { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.LiveRecordLimit"/>.
/// </summary>
public TimeSpan? LiveRecordLimit { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.TaskStartAt"/>.
/// </summary>
public DateTime? TaskStartAt { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.SkipMerge"/>.
/// </summary>
public bool SkipMerge { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.BinaryMerge"/>.
/// </summary>
public bool BinaryMerge { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.ForceAnsiConsole"/>.
/// </summary>
public bool ForceAnsiConsole { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.NoAnsiColor"/>.
/// </summary>
public bool NoAnsiColor { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.UseFFmpegConcatDemuxer"/>.
/// </summary>
public bool UseFFmpegConcatDemuxer { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.DelAfterDone"/>.
/// </summary>
public bool DelAfterDone { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.AutoSubtitleFix"/>.
/// </summary>
public bool AutoSubtitleFix { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.CheckSegmentsCount"/>.
/// </summary>
public bool CheckSegmentsCount { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.SkipDownload"/>.
/// </summary>
public bool SkipDownload { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.WriteMetaJson"/>.
/// </summary>
public bool WriteMetaJson { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.AppendUrlParams"/>.
/// </summary>
public bool AppendUrlParams { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.MP4RealTimeDecryption"/>.
/// </summary>
public bool MP4RealTimeDecryption { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.UseShakaPackager"/>.
/// </summary>
[Obsolete("Use DecryptionEngine instead")]
public bool UseShakaPackager { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.DecryptionEngine"/>.
/// </summary>
public DecryptEngine DecryptionEngine { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.MuxAfterDone"/>.
/// </summary>
public bool MuxAfterDone { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.ConcurrentDownload"/>.
/// </summary>
public bool ConcurrentDownload { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.LiveRealTimeMerge"/>.
/// </summary>
public bool LiveRealTimeMerge { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.LiveKeepSegments"/>.
/// </summary>
public bool LiveKeepSegments { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.LivePerformAsVod"/>.
/// </summary>
public bool LivePerformAsVod { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.UseSystemProxy"/>.
/// </summary>
public bool UseSystemProxy { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.SubtitleFormat"/>.
/// </summary>
public SubtitleFormat SubtitleFormat { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.TmpDir"/>.
/// </summary>
public string? TmpDir { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.SaveDir"/>.
/// </summary>
public string? SaveDir { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.SaveName"/>.
/// </summary>
public string? SaveName { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.SavePattern"/>.
/// </summary>
public string? SavePattern { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.UILanguage"/>.
/// </summary>
public string? UILanguage { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.DecryptionBinaryPath"/>.
/// </summary>
public string? DecryptionBinaryPath { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.FFmpegBinaryPath"/>.
/// </summary>
public string? FFmpegBinaryPath { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.MkvmergeBinaryPath"/>.
/// </summary>
public string? MkvmergeBinaryPath { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.MuxImports"/>.
/// </summary>
public List<OutputFile>? MuxImports { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.VideoFilter"/>.
/// </summary>
public StreamFilter? VideoFilter { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.DropVideoFilter"/>.
/// </summary>
public StreamFilter? DropVideoFilter { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.AudioFilter"/>.
/// </summary>
public StreamFilter? AudioFilter { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.DropAudioFilter"/>.
/// </summary>
public StreamFilter? DropAudioFilter { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.SubtitleFilter"/>.
/// </summary>
public StreamFilter? SubtitleFilter { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.DropSubtitleFilter"/>.
/// </summary>
public StreamFilter? DropSubtitleFilter { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.CustomHLSMethod"/>.
/// </summary>
public EncryptMethod? CustomHLSMethod { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.CustomHLSKey"/>.
/// </summary>
public byte[]? CustomHLSKey { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.CustomHLSIv"/>.
/// </summary>
public byte[]? CustomHLSIv { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.CustomProxy"/>.
/// </summary>
public WebProxy? CustomProxy { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.CustomRange"/>.
/// </summary>
public CustomRange? CustomRange { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.LiveWaitTime"/>.
/// </summary>
public int? LiveWaitTime { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.LiveTakeCount"/>.
/// </summary>
public int LiveTakeCount { get; set; }
public MuxOptions? MuxOptions { get; set; }
// public bool LiveWriteHLS { get; set; } = true;
/// <summary>
/// See: <see cref="CommandInvoker.LivePipeMux"/>.
/// </summary>
public bool LivePipeMux { get; set; }
/// <summary>
/// See: <see cref="CommandInvoker.LiveFixVttByAudio"/>.
/// </summary>
public bool LiveFixVttByAudio { get; set; }
}

25
src/N_m3u8DL-RE/Config/DownloaderConfig.cs Normal file
View File

@ -0,0 +1,25 @@
using N_m3u8DL_RE.CommandLine;
namespace N_m3u8DL_RE.Config;
internal class DownloaderConfig
{
public required MyOption MyOptions { get; set; }
/// <summary>
/// 前置阶段生成的文件夹名
/// </summary>
public required string DirPrefix { get; set; }
/// <summary>
/// 文件名模板
/// </summary>
public string? SavePattern { get; set; }
/// <summary>
/// 校验响应头的文件大小和实际大小
/// </summary>
public bool CheckContentLength { get; set; } = true;
/// <summary>
/// 请求头
/// </summary>
public Dictionary<string, string> Headers { get; set; } = new Dictionary<string, string>();
}

38
src/N_m3u8DL-RE/Crypto/AESUtil.cs Normal file
View File

@ -0,0 +1,38 @@
using System.Security.Cryptography;
namespace N_m3u8DL_RE.Crypto;
internal static class AESUtil
{
/// <summary>
/// AES-128解密，解密后原地替换文件
/// </summary>
/// <param name="filePath"></param>
/// <param name="keyByte"></param>
/// <param name="ivByte"></param>
/// <param name="mode"></param>
/// <param name="padding"></param>
public static void AES128Decrypt(string filePath, byte[] keyByte, byte[] ivByte, CipherMode mode = CipherMode.CBC, PaddingMode padding = PaddingMode.PKCS7)
{
var fileBytes = File.ReadAllBytes(filePath);
var decrypted = AES128Decrypt(fileBytes, keyByte, ivByte, mode, padding);
File.WriteAllBytes(filePath, decrypted);
}
public static byte[] AES128Decrypt(byte[] encryptedBuff, byte[] keyByte, byte[] ivByte, CipherMode mode = CipherMode.CBC, PaddingMode padding = PaddingMode.PKCS7)
{
byte[] inBuff = encryptedBuff;
Aes dcpt = Aes.Create();
dcpt.BlockSize = 128;
dcpt.KeySize = 128;
dcpt.Key = keyByte;
dcpt.IV = ivByte;
dcpt.Mode = mode;
dcpt.Padding = padding;
ICryptoTransform cTransform = dcpt.CreateDecryptor();
byte[] resultArray = cTransform.TransformFinalBlock(inBuff, 0, inBuff.Length);
return resultArray;
}
}

661
src/N_m3u8DL-RE/Crypto/CSChaCha20.cs Normal file
View File

@ -0,0 +1,661 @@
/*
* Copyright (c) 2015, 2018 Scott Bennett
* (c) 2018-2021 Kaarlo Räihä
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
using System;
using System.IO;
using System.Text;
using System.Threading.Tasks;
using System.Runtime.CompilerServices; // For MethodImplOptions.AggressiveInlining
namespace CSChaCha20
{
/// <summary>
/// Class that can be used for ChaCha20 encryption / decryption
/// </summary>
public sealed class ChaCha20 : IDisposable
{
/// <summary>
/// Only allowed key lenght in bytes
/// </summary>
public const int allowedKeyLength = 32;
/// <summary>
/// Only allowed nonce lenght in bytes
/// </summary>
public const int allowedNonceLength = 12;
/// <summary>
/// How many bytes are processed per loop
/// </summary>
public const int processBytesAtTime = 64;
private const int stateLength = 16;
/// <summary>
/// The ChaCha20 state (aka "context")
/// </summary>
private readonly uint[] state = new uint[stateLength];
/// <summary>
/// Determines if the objects in this class have been disposed of. Set to true by the Dispose() method.
/// </summary>
private bool isDisposed = false;
/// <summary>
/// Set up a new ChaCha20 state. The lengths of the given parameters are checked before encryption happens.
/// </summary>
/// <remarks>
/// See <a href="https://tools.ietf.org/html/rfc7539#page-10">ChaCha20 Spec Section 2.4</a> for a detailed description of the inputs.
/// </remarks>
/// <param name="key">
/// A 32-byte (256-bit) key, treated as a concatenation of eight 32-bit little-endian integers
/// </param>
/// <param name="nonce">
/// A 12-byte (96-bit) nonce, treated as a concatenation of three 32-bit little-endian integers
/// </param>
/// <param name="counter">
/// A 4-byte (32-bit) block counter, treated as a 32-bit little-endian integer
/// </param>
public ChaCha20(byte[] key, byte[] nonce, uint counter)
{
this.KeySetup(key);
this.IvSetup(nonce, counter);
}
#if NET6_0_OR_GREATER
/// <summary>
/// Set up a new ChaCha20 state. The lengths of the given parameters are checked before encryption happens.
/// </summary>
/// <remarks>
/// See <a href="https://tools.ietf.org/html/rfc7539#page-10">ChaCha20 Spec Section 2.4</a> for a detailed description of the inputs.
/// </remarks>
/// <param name="key">A 32-byte (256-bit) key, treated as a concatenation of eight 32-bit little-endian integers</param>
/// <param name="nonce">A 12-byte (96-bit) nonce, treated as a concatenation of three 32-bit little-endian integers</param>
/// <param name="counter">A 4-byte (32-bit) block counter, treated as a 32-bit little-endian integer</param>
public ChaCha20(ReadOnlySpan<byte> key, ReadOnlySpan<byte> nonce, uint counter)
{
this.KeySetup(key.ToArray());
this.IvSetup(nonce.ToArray(), counter);
}
#endif // NET6_0_OR_GREATER
/// <summary>
/// The ChaCha20 state (aka "context"). Read-Only.
/// </summary>
public uint[] State
{
get
{
return this.state;
}
}
// These are the same constants defined in the reference implementation.
// http://cr.yp.to/streamciphers/timings/estreambench/submissions/salsa20/chacha8/ref/chacha.c
private static readonly byte[] sigma = Encoding.ASCII.GetBytes("expand 32-byte k");
private static readonly byte[] tau = Encoding.ASCII.GetBytes("expand 16-byte k");
/// <summary>
/// Set up the ChaCha state with the given key. A 32-byte key is required and enforced.
/// </summary>
/// <param name="key">
/// A 32-byte (256-bit) key, treated as a concatenation of eight 32-bit little-endian integers
/// </param>
private void KeySetup(byte[] key)
{
if (key == null)
{
throw new ArgumentNullException("Key is null");
}
if (key.Length != allowedKeyLength)
{
throw new ArgumentException($"Key length must be {allowedKeyLength}. Actual: {key.Length}");
}
state[4] = Util.U8To32Little(key, 0);
state[5] = Util.U8To32Little(key, 4);
state[6] = Util.U8To32Little(key, 8);
state[7] = Util.U8To32Little(key, 12);
byte[] constants = (key.Length == allowedKeyLength) ? sigma : tau;
int keyIndex = key.Length - 16;
state[8] = Util.U8To32Little(key, keyIndex + 0);
state[9] = Util.U8To32Little(key, keyIndex + 4);
state[10] = Util.U8To32Little(key, keyIndex + 8);
state[11] = Util.U8To32Little(key, keyIndex + 12);
state[0] = Util.U8To32Little(constants, 0);
state[1] = Util.U8To32Little(constants, 4);
state[2] = Util.U8To32Little(constants, 8);
state[3] = Util.U8To32Little(constants, 12);
}
/// <summary>
/// Set up the ChaCha state with the given nonce (aka Initialization Vector or IV) and block counter. A 12-byte nonce and a 4-byte counter are required.
/// </summary>
/// <param name="nonce">
/// A 12-byte (96-bit) nonce, treated as a concatenation of three 32-bit little-endian integers
/// </param>
/// <param name="counter">
/// A 4-byte (32-bit) block counter, treated as a 32-bit little-endian integer
/// </param>
private void IvSetup(byte[] nonce, uint counter)
{
if (nonce == null)
{
// There has already been some state set up. Clear it before exiting.
Dispose();
throw new ArgumentNullException("Nonce is null");
}
if (nonce.Length != allowedNonceLength)
{
// There has already been some state set up. Clear it before exiting.
Dispose();
throw new ArgumentException($"Nonce length must be {allowedNonceLength}. Actual: {nonce.Length}");
}
state[12] = counter;
state[13] = Util.U8To32Little(nonce, 0);
state[14] = Util.U8To32Little(nonce, 4);
state[15] = Util.U8To32Little(nonce, 8);
}
#region Encryption methods
/// <summary>
/// Encrypt arbitrary-length byte array (input), writing the resulting byte array to preallocated output buffer.
/// </summary>
/// <remarks>Since this is symmetric operation, it doesn't really matter if you use Encrypt or Decrypt method</remarks>
/// <param name="output">Output byte array, must have enough bytes</param>
/// <param name="input">Input byte array</param>
/// <param name="numBytes">Number of bytes to encrypt</param>
public void EncryptBytes(byte[] output, byte[] input, int numBytes)
{
this.WorkBytes(output, input, numBytes);
}
/// <summary>
/// Encrypt arbitrary-length byte stream (input), writing the resulting bytes to another stream (output)
/// </summary>
/// <param name="output">Output stream</param>
/// <param name="input">Input stream</param>
/// <param name="howManyBytesToProcessAtTime">How many bytes to read and write at time, default is 1024</param>
public void EncryptStream(Stream output, Stream input, int howManyBytesToProcessAtTime = 1024)
{
this.WorkStreams(output, input, howManyBytesToProcessAtTime);
}
/// <summary>
/// Async encrypt arbitrary-length byte stream (input), writing the resulting bytes to another stream (output)
/// </summary>
/// <param name="output">Output stream</param>
/// <param name="input">Input stream</param>
/// <param name="howManyBytesToProcessAtTime">How many bytes to read and write at time, default is 1024</param>
public async Task EncryptStreamAsync(Stream output, Stream input, int howManyBytesToProcessAtTime = 1024)
{
await this.WorkStreamsAsync(output, input, howManyBytesToProcessAtTime);
}
/// <summary>
/// Encrypt arbitrary-length byte array (input), writing the resulting byte array to preallocated output buffer.
/// </summary>
/// <remarks>Since this is symmetric operation, it doesn't really matter if you use Encrypt or Decrypt method</remarks>
/// <param name="output">Output byte array, must have enough bytes</param>
/// <param name="input">Input byte array</param>
public void EncryptBytes(byte[] output, byte[] input)
{
this.WorkBytes(output, input, input.Length);
}
/// <summary>
/// Encrypt arbitrary-length byte array (input), writing the resulting byte array that is allocated by method.
/// </summary>
/// <remarks>Since this is symmetric operation, it doesn't really matter if you use Encrypt or Decrypt method</remarks>
/// <param name="input">Input byte array</param>
/// <param name="numBytes">Number of bytes to encrypt</param>
/// <returns>Byte array that contains encrypted bytes</returns>
public byte[] EncryptBytes(byte[] input, int numBytes)
{
byte[] returnArray = new byte[numBytes];
this.WorkBytes(returnArray, input, numBytes);
return returnArray;
}
/// <summary>
/// Encrypt arbitrary-length byte array (input), writing the resulting byte array that is allocated by method.
/// </summary>
/// <remarks>Since this is symmetric operation, it doesn't really matter if you use Encrypt or Decrypt method</remarks>
/// <param name="input">Input byte array</param>
/// <returns>Byte array that contains encrypted bytes</returns>
public byte[] EncryptBytes(byte[] input)
{
byte[] returnArray = new byte[input.Length];
this.WorkBytes(returnArray, input, input.Length);
return returnArray;
}
/// <summary>
/// Encrypt string as UTF8 byte array, returns byte array that is allocated by method.
/// </summary>
/// <remarks>Here you can NOT swap encrypt and decrypt methods, because of bytes-string transform</remarks>
/// <param name="input">Input string</param>
/// <returns>Byte array that contains encrypted bytes</returns>
public byte[] EncryptString(string input)
{
byte[] utf8Bytes = System.Text.Encoding.UTF8.GetBytes(input);
byte[] returnArray = new byte[utf8Bytes.Length];
this.WorkBytes(returnArray, utf8Bytes, utf8Bytes.Length);
return returnArray;
}
#endregion // Encryption methods
#region // Decryption methods
/// <summary>
/// Decrypt arbitrary-length byte array (input), writing the resulting byte array to the output buffer.
/// </summary>
/// <remarks>Since this is symmetric operation, it doesn't really matter if you use Encrypt or Decrypt method</remarks>
/// <param name="output">Output byte array</param>
/// <param name="input">Input byte array</param>
/// <param name="numBytes">Number of bytes to decrypt</param>
public void DecryptBytes(byte[] output, byte[] input, int numBytes)
{
this.WorkBytes(output, input, numBytes);
}
/// <summary>
/// Decrypt arbitrary-length byte stream (input), writing the resulting bytes to another stream (output)
/// </summary>
/// <param name="output">Output stream</param>
/// <param name="input">Input stream</param>
/// <param name="howManyBytesToProcessAtTime">How many bytes to read and write at time, default is 1024</param>
public void DecryptStream(Stream output, Stream input, int howManyBytesToProcessAtTime = 1024)
{
this.WorkStreams(output, input, howManyBytesToProcessAtTime);
}
/// <summary>
/// Async decrypt arbitrary-length byte stream (input), writing the resulting bytes to another stream (output)
/// </summary>
/// <param name="output">Output stream</param>
/// <param name="input">Input stream</param>
/// <param name="howManyBytesToProcessAtTime">How many bytes to read and write at time, default is 1024</param>
public async Task DecryptStreamAsync(Stream output, Stream input, int howManyBytesToProcessAtTime = 1024)
{
await this.WorkStreamsAsync(output, input, howManyBytesToProcessAtTime);
}
/// <summary>
/// Decrypt arbitrary-length byte array (input), writing the resulting byte array to preallocated output buffer.
/// </summary>
/// <remarks>Since this is symmetric operation, it doesn't really matter if you use Encrypt or Decrypt method</remarks>
/// <param name="output">Output byte array, must have enough bytes</param>
/// <param name="input">Input byte array</param>
public void DecryptBytes(byte[] output, byte[] input)
{
WorkBytes(output, input, input.Length);
}
/// <summary>
/// Decrypt arbitrary-length byte array (input), writing the resulting byte array that is allocated by method.
/// </summary>
/// <remarks>Since this is symmetric operation, it doesn't really matter if you use Encrypt or Decrypt method</remarks>
/// <param name="input">Input byte array</param>
/// <param name="numBytes">Number of bytes to encrypt</param>
/// <returns>Byte array that contains decrypted bytes</returns>
public byte[] DecryptBytes(byte[] input, int numBytes)
{
byte[] returnArray = new byte[numBytes];
WorkBytes(returnArray, input, numBytes);
return returnArray;
}
/// <summary>
/// Decrypt arbitrary-length byte array (input), writing the resulting byte array that is allocated by method.
/// </summary>
/// <remarks>Since this is symmetric operation, it doesn't really matter if you use Encrypt or Decrypt method</remarks>
/// <param name="input">Input byte array</param>
/// <returns>Byte array that contains decrypted bytes</returns>
public byte[] DecryptBytes(byte[] input)
{
byte[] returnArray = new byte[input.Length];
WorkBytes(returnArray, input, input.Length);
return returnArray;
}
/// <summary>
/// Decrypt UTF8 byte array to string.
/// </summary>
/// <remarks>Here you can NOT swap encrypt and decrypt methods, because of bytes-string transform</remarks>
/// <param name="input">Byte array</param>
/// <returns>Byte array that contains encrypted bytes</returns>
public string DecryptUTF8ByteArray(byte[] input)
{
byte[] tempArray = new byte[input.Length];
WorkBytes(tempArray, input, input.Length);
return System.Text.Encoding.UTF8.GetString(tempArray);
}
#endregion // Decryption methods
private void WorkStreams(Stream output, Stream input, int howManyBytesToProcessAtTime = 1024)
{
int readBytes;
byte[] inputBuffer = new byte[howManyBytesToProcessAtTime];
byte[] outputBuffer = new byte[howManyBytesToProcessAtTime];
while ((readBytes = input.Read(inputBuffer, 0, howManyBytesToProcessAtTime)) > 0)
{
// Encrypt or decrypt
WorkBytes(output: outputBuffer, input: inputBuffer, numBytes: readBytes);
// Write buffer
output.Write(outputBuffer, 0, readBytes);
}
}
private async Task WorkStreamsAsync(Stream output, Stream input, int howManyBytesToProcessAtTime = 1024)
{
byte[] readBytesBuffer = new byte[howManyBytesToProcessAtTime];
byte[] writeBytesBuffer = new byte[howManyBytesToProcessAtTime];
int howManyBytesWereRead = await input.ReadAsync(readBytesBuffer, 0, howManyBytesToProcessAtTime);
while (howManyBytesWereRead > 0)
{
// Encrypt or decrypt
WorkBytes(output: writeBytesBuffer, input: readBytesBuffer, numBytes: howManyBytesWereRead);
// Write
await output.WriteAsync(writeBytesBuffer, 0, howManyBytesWereRead);
// Read more
howManyBytesWereRead = await input.ReadAsync(readBytesBuffer, 0, howManyBytesToProcessAtTime);
}
}
/// <summary>
/// Encrypt or decrypt an arbitrary-length byte array (input), writing the resulting byte array to the output buffer. The number of bytes to read from the input buffer is determined by numBytes.
/// </summary>
/// <param name="output">Output byte array</param>
/// <param name="input">Input byte array</param>
/// <param name="numBytes">How many bytes to process</param>
private void WorkBytes(byte[] output, byte[] input, int numBytes)
{
if (isDisposed)
{
throw new ObjectDisposedException("state", "The ChaCha state has been disposed");
}
if (input == null)
{
throw new ArgumentNullException("input", "Input cannot be null");
}
if (output == null)
{
throw new ArgumentNullException("output", "Output cannot be null");
}
if (numBytes < 0 || numBytes > input.Length)
{
throw new ArgumentOutOfRangeException("numBytes", "The number of bytes to read must be between [0..input.Length]");
}
if (output.Length < numBytes)
{
throw new ArgumentOutOfRangeException("output", $"Output byte array should be able to take at least {numBytes}");
}
uint[] x = new uint[stateLength]; // Working buffer
byte[] tmp = new byte[processBytesAtTime]; // Temporary buffer
int offset = 0;
while (numBytes > 0)
{
// Copy state to working buffer
Buffer.BlockCopy(this.state, 0, x, 0, stateLength * sizeof(uint));
for (int i = 0; i < 10; i++)
{
QuarterRound(x, 0, 4, 8, 12);
QuarterRound(x, 1, 5, 9, 13);
QuarterRound(x, 2, 6, 10, 14);
QuarterRound(x, 3, 7, 11, 15);
QuarterRound(x, 0, 5, 10, 15);
QuarterRound(x, 1, 6, 11, 12);
QuarterRound(x, 2, 7, 8, 13);
QuarterRound(x, 3, 4, 9, 14);
}
for (int i = 0; i < stateLength; i++)
{
Util.ToBytes(tmp, Util.Add(x[i], this.state[i]), 4 * i);
}
this.state[12] = Util.AddOne(state[12]);
if (this.state[12] <= 0)
{
/* Stopping at 2^70 bytes per nonce is the user's responsibility */
this.state[13] = Util.AddOne(state[13]);
}
// In case these are last bytes
if (numBytes <= processBytesAtTime)
{
for (int i = 0; i < numBytes; i++)
{
output[i + offset] = (byte)(input[i + offset] ^ tmp[i]);
}
return;
}
for (int i = 0; i < processBytesAtTime; i++)
{
output[i + offset] = (byte)(input[i + offset] ^ tmp[i]);
}
numBytes -= processBytesAtTime;
offset += processBytesAtTime;
}
}
/// <summary>
/// The ChaCha Quarter Round operation. It operates on four 32-bit unsigned integers within the given buffer at indices a, b, c, and d.
/// </summary>
/// <remarks>
/// The ChaCha state does not have four integer numbers: it has 16. So the quarter-round operation works on only four of them -- hence the name. Each quarter round operates on four predetermined numbers in the ChaCha state.
/// See <a href="https://tools.ietf.org/html/rfc7539#page-4">ChaCha20 Spec Sections 2.1 - 2.2</a>.
/// </remarks>
/// <param name="x">A ChaCha state (vector). Must contain 16 elements.</param>
/// <param name="a">Index of the first number</param>
/// <param name="b">Index of the second number</param>
/// <param name="c">Index of the third number</param>
/// <param name="d">Index of the fourth number</param>
private static void QuarterRound(uint[] x, uint a, uint b, uint c, uint d)
{
x[a] = Util.Add(x[a], x[b]);
x[d] = Util.Rotate(Util.XOr(x[d], x[a]), 16);
x[c] = Util.Add(x[c], x[d]);
x[b] = Util.Rotate(Util.XOr(x[b], x[c]), 12);
x[a] = Util.Add(x[a], x[b]);
x[d] = Util.Rotate(Util.XOr(x[d], x[a]), 8);
x[c] = Util.Add(x[c], x[d]);
x[b] = Util.Rotate(Util.XOr(x[b], x[c]), 7);
}
#region Destructor and Disposer
/// <summary>
/// Clear and dispose of the internal state. The finalizer is only called if Dispose() was never called on this cipher.
/// </summary>
~ChaCha20()
{
Dispose(false);
}
/// <summary>
/// Clear and dispose of the internal state. Also request the GC not to call the finalizer, because all cleanup has been taken care of.
/// </summary>
public void Dispose()
{
Dispose(true);
/*
* The Garbage Collector does not need to invoke the finalizer because Dispose(bool) has already done all the cleanup needed.
*/
GC.SuppressFinalize(this);
}
/// <summary>
/// This method should only be invoked from Dispose() or the finalizer. This handles the actual cleanup of the resources.
/// </summary>
/// <param name="disposing">
/// Should be true if called by Dispose(); false if called by the finalizer
/// </param>
private void Dispose(bool disposing)
{
if (!isDisposed)
{
if (disposing)
{
/* Cleanup managed objects by calling their Dispose() methods */
}
/* Cleanup any unmanaged objects here */
Array.Clear(state, 0, stateLength);
}
isDisposed = true;
}
#endregion // Destructor and Disposer
}
/// <summary>
/// Utilities that are used during compression
/// </summary>
public static class Util
{
/// <summary>
/// n-bit left rotation operation (towards the high bits) for 32-bit integers.
/// </summary>
/// <param name="v"></param>
/// <param name="c"></param>
/// <returns>The result of (v LEFTSHIFT c)</returns>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint Rotate(uint v, int c)
{
unchecked
{
return (v << c) | (v >> (32 - c));
}
}
/// <summary>
/// Unchecked integer exclusive or (XOR) operation.
/// </summary>
/// <param name="v"></param>
/// <param name="w"></param>
/// <returns>The result of (v XOR w)</returns>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint XOr(uint v, uint w)
{
return unchecked(v ^ w);
}
/// <summary>
/// Unchecked integer addition. The ChaCha spec defines certain operations to use 32-bit unsigned integer addition modulo 2^32.
/// </summary>
/// <remarks>
/// See <a href="https://tools.ietf.org/html/rfc7539#page-4">ChaCha20 Spec Section 2.1</a>.
/// </remarks>
/// <param name="v"></param>
/// <param name="w"></param>
/// <returns>The result of (v + w) modulo 2^32</returns>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint Add(uint v, uint w)
{
return unchecked(v + w);
}
/// <summary>
/// Add 1 to the input parameter using unchecked integer addition. The ChaCha spec defines certain operations to use 32-bit unsigned integer addition modulo 2^32.
/// </summary>
/// <remarks>
/// See <a href="https://tools.ietf.org/html/rfc7539#page-4">ChaCha20 Spec Section 2.1</a>.
/// </remarks>
/// <param name="v"></param>
/// <returns>The result of (v + 1) modulo 2^32</returns>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint AddOne(uint v)
{
return unchecked(v + 1);
}
/// <summary>
/// Convert four bytes of the input buffer into an unsigned 32-bit integer, beginning at the inputOffset.
/// </summary>
/// <param name="p"></param>
/// <param name="inputOffset"></param>
/// <returns>An unsigned 32-bit integer</returns>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint U8To32Little(byte[] p, int inputOffset)
{
unchecked
{
return ((uint)p[inputOffset]
| ((uint)p[inputOffset + 1] << 8)
| ((uint)p[inputOffset + 2] << 16)
| ((uint)p[inputOffset + 3] << 24));
}
}
/// <summary>
/// Serialize the input integer into the output buffer. The input integer will be split into 4 bytes and put into four sequential places in the output buffer, starting at the outputOffset.
/// </summary>
/// <param name="output"></param>
/// <param name="input"></param>
/// <param name="outputOffset"></param>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void ToBytes(byte[] output, uint input, int outputOffset)
{
unchecked
{
output[outputOffset] = (byte)input;
output[outputOffset + 1] = (byte)(input >> 8);
output[outputOffset + 2] = (byte)(input >> 16);
output[outputOffset + 3] = (byte)(input >> 24);
}
}
}
}

37
src/N_m3u8DL-RE/Crypto/ChaCha20Util.cs Normal file
View File

@ -0,0 +1,37 @@
using CSChaCha20;
namespace N_m3u8DL_RE.Crypto;
internal static class ChaCha20Util
{
public static byte[] DecryptPer1024Bytes(byte[] encryptedBuff, byte[] keyBytes, byte[] nonceBytes)
{
if (keyBytes.Length != 32)
throw new Exception("Key must be 32 bytes!");
if (nonceBytes.Length != 12 && nonceBytes.Length != 8)
throw new Exception("Key must be 12 or 8 bytes!");
if (nonceBytes.Length == 8)
nonceBytes = (new byte[4] { 0, 0, 0, 0 }).Concat(nonceBytes).ToArray();
var decStream = new MemoryStream();
using BinaryReader reader = new BinaryReader(new MemoryStream(encryptedBuff));
using (BinaryWriter writer = new BinaryWriter(decStream))
while (true)
{
var buffer = reader.ReadBytes(1024);
byte[] dec = new byte[buffer.Length];
if (buffer.Length > 0)
{
ChaCha20 forDecrypting = new ChaCha20(keyBytes, nonceBytes, 0);
forDecrypting.DecryptBytes(dec, buffer);
writer.Write(dec, 0, dec.Length);
}
else
{
break;
}
}
return decStream.ToArray();
}
}

25
src/N_m3u8DL-RE/Directory.Build.props Normal file
View File

@ -0,0 +1,25 @@
<Project>
<PropertyGroup>
<IlcOptimizationPreference>Speed</IlcOptimizationPreference>
<IlcFoldIdenticalMethodBodies>true</IlcFoldIdenticalMethodBodies>
<StaticallyLinked Condition="$(RuntimeIdentifier.StartsWith('win'))">true</StaticallyLinked>
<TrimMode>full</TrimMode>
<TrimmerDefaultAction>link</TrimmerDefaultAction>
<IlcTrimMetadata>true</IlcTrimMetadata>
<IlcGenerateStackTraceData>true</IlcGenerateStackTraceData>
<SatelliteResourceLanguages>zh-CN;zh-TW;en-US</SatelliteResourceLanguages>
<PublishAot>true</PublishAot>
<StripSymbols>true</StripSymbols>
<ObjCopyName Condition="'$(RuntimeIdentifier)' == 'linux-arm64' or '$(RuntimeIdentifier)' == 'linux-musl-arm64'">aarch64-linux-gnu-objcopy</ObjCopyName>
</PropertyGroup>
<!--<ItemGroup Condition="'$(PublishAot)' == 'true' and '$(RuntimeIdentifier)' != 'win-arm64' and '$(RuntimeIdentifier)' != 'linux-arm64' and '$(RuntimeIdentifier)' != 'osx-arm64' and '$(RuntimeIdentifier)' != 'osx-x64'">
<PackageReference Include="PublishAotCompressed" Version="1.0.3" />
</ItemGroup>-->
<ItemGroup>
<RdXmlFile Include="rd.xml" />
</ItemGroup>
</Project>

242
src/N_m3u8DL-RE/DownloadManager/HTTPLiveRecordManager.cs Normal file
View File

@ -0,0 +1,242 @@
using N_m3u8DL_RE.Column;
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Config;
using N_m3u8DL_RE.Downloader;
using N_m3u8DL_RE.Entity;
using N_m3u8DL_RE.Parser;
using N_m3u8DL_RE.Util;
using Spectre.Console;
using System.Collections.Concurrent;
using System.Text;
namespace N_m3u8DL_RE.DownloadManager;
internal class HTTPLiveRecordManager
{
IDownloader Downloader;
DownloaderConfig DownloaderConfig;
StreamExtractor StreamExtractor;
List<StreamSpec> SelectedSteams;
List<OutputFile> OutputFiles = [];
DateTime NowDateTime;
DateTime? PublishDateTime;
bool STOP_FLAG = false;
bool READ_IFO = false;
ConcurrentDictionary<int, int> RecordingDurDic = new(); // 已录制时长
ConcurrentDictionary<int, double> RecordingSizeDic = new(); // 已录制大小
CancellationTokenSource CancellationTokenSource = new(); // 取消Wait
List<byte> InfoBuffer = new List<byte>(188 * 5000); // 5000个分包中解析信息，没有就算了
public HTTPLiveRecordManager(DownloaderConfig downloaderConfig, List<StreamSpec> selectedSteams, StreamExtractor streamExtractor)
{
this.DownloaderConfig = downloaderConfig;
Downloader = new SimpleDownloader(DownloaderConfig);
NowDateTime = DateTime.Now;
PublishDateTime = selectedSteams.FirstOrDefault()?.PublishTime;
StreamExtractor = streamExtractor;
SelectedSteams = selectedSteams;
}
private async Task<bool> RecordStreamAsync(StreamSpec streamSpec, ProgressTask task, SpeedContainer speedContainer)
{
task.MaxValue = 1;
task.StartTask();
var name = streamSpec.ToShortString();
var dirName = $"{DownloaderConfig.MyOptions.SaveName ?? NowDateTime.ToString("yyyy-MM-dd_HH-mm-ss")}_{task.Id}_{OtherUtil.GetValidFileName(streamSpec.GroupId ?? "", "-")}_{streamSpec.Codecs}_{streamSpec.Bandwidth}_{streamSpec.Language}";
var saveDir = DownloaderConfig.MyOptions.SaveDir ?? Environment.CurrentDirectory;
var saveName = DownloaderConfig.MyOptions.SaveName != null ? $"{DownloaderConfig.MyOptions.SaveName}.{streamSpec.Language}".TrimEnd('.') : dirName;
Logger.Debug($"dirName: {dirName}; saveDir: {saveDir}; saveName: {saveName}");
// 创建文件夹
if (!Directory.Exists(saveDir)) Directory.CreateDirectory(saveDir);
using var request = new HttpRequestMessage(HttpMethod.Get, new Uri(streamSpec.Url));
request.Headers.ConnectionClose = false;
foreach (var item in DownloaderConfig.Headers)
{
request.Headers.TryAddWithoutValidation(item.Key, item.Value);
}
Logger.Debug(request.Headers.ToString());
using var response = await HTTPUtil.AppHttpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, CancellationTokenSource.Token);
response.EnsureSuccessStatusCode();
var output = Path.Combine(saveDir, saveName + ".ts");
using var stream = new FileStream(output, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.Read);
using var responseStream = await response.Content.ReadAsStreamAsync(CancellationTokenSource.Token);
var buffer = new byte[16 * 1024];
var size = 0;
// 计时器
_ = TimeCounterAsync();
// 读取INFO
_ = ReadInfoAsync();
try
{
while ((size = await responseStream.ReadAsync(buffer, CancellationTokenSource.Token)) > 0)
{
if (!READ_IFO && InfoBuffer.Count < 188 * 5000)
{
InfoBuffer.AddRange(buffer);
}
speedContainer.Add(size);
RecordingSizeDic[task.Id] += size;
await stream.WriteAsync(buffer, 0, size);
}
}
catch (OperationCanceledException oce) when (oce.CancellationToken == CancellationTokenSource.Token)
{
;
}
Logger.InfoMarkUp("File Size: " + GlobalUtil.FormatFileSize(RecordingSizeDic[task.Id]));
return true;
}
public async Task ReadInfoAsync()
{
while (!STOP_FLAG && !READ_IFO)
{
await Task.Delay(200);
if (InfoBuffer.Count < 188 * 5000) continue;
ushort ConvertToUint16(IEnumerable<byte> bytes)
{
if (BitConverter.IsLittleEndian)
bytes = bytes.Reverse();
return BitConverter.ToUInt16(bytes.ToArray());
}
var data = InfoBuffer.ToArray();
var programId = "";
var serviceProvider = "";
var serviceName = "";
for (int i = 0; i < data.Length; i++)
{
if (data[i] == 0x47 && (i + 188) < data.Length && data[i + 188] == 0x47)
{
var tsData = data.Skip(i).Take(188);
var tsHeaderInt = BitConverter.ToUInt32(BitConverter.IsLittleEndian ? tsData.Take(4).Reverse().ToArray() : tsData.Take(4).ToArray(), 0);
var pid = (tsHeaderInt & 0x1fff00) >> 8;
var tsPayload = tsData.Skip(4);
// PAT
if (pid == 0x0000)
{
programId = ConvertToUint16(tsPayload.Skip(9).Take(2)).ToString();
}
// SDT, BAT, ST
else if (pid == 0x0011)
{
var tableId = (int)tsPayload.Skip(1).First();
// Current TS Info
if (tableId == 0x42)
{
var sectionLength = ConvertToUint16(tsPayload.Skip(2).Take(2)) & 0xfff;
var sectionData = tsPayload.Skip(4).Take(sectionLength);
var dscripData = sectionData.Skip(8);
var descriptorsLoopLength = (ConvertToUint16(dscripData.Skip(3).Take(2))) & 0xfff;
var descriptorsData = dscripData.Skip(5).Take(descriptorsLoopLength);
var serviceProviderLength = (int)descriptorsData.Skip(3).First();
serviceProvider = Encoding.UTF8.GetString(descriptorsData.Skip(4).Take(serviceProviderLength).ToArray());
var serviceNameLength = (int)descriptorsData.Skip(4 + serviceProviderLength).First();
serviceName = Encoding.UTF8.GetString(descriptorsData.Skip(5 + serviceProviderLength).Take(serviceNameLength).ToArray());
}
}
if (programId != "" && (serviceName != "" || serviceProvider != ""))
break;
}
}
if (!string.IsNullOrEmpty(programId))
{
Logger.InfoMarkUp($"Program Id: [cyan]{programId.EscapeMarkup()}[/]");
if (!string.IsNullOrEmpty(serviceName)) Logger.InfoMarkUp($"Service Name: [cyan]{serviceName.EscapeMarkup()}[/]");
if (!string.IsNullOrEmpty(serviceProvider)) Logger.InfoMarkUp($"Service Provider: [cyan]{serviceProvider.EscapeMarkup()}[/]");
READ_IFO = true;
}
}
}
public async Task TimeCounterAsync()
{
while (!STOP_FLAG)
{
await Task.Delay(1000);
RecordingDurDic[0]++;
// 检测时长限制
if (RecordingDurDic.All(d => d.Value >= DownloaderConfig.MyOptions.LiveRecordLimit?.TotalSeconds))
{
Logger.WarnMarkUp($"[darkorange3_1]{ResString.liveLimitReached}[/]");
STOP_FLAG = true;
CancellationTokenSource.Cancel();
}
}
}
public async Task<bool> StartRecordAsync()
{
ConcurrentDictionary<int, SpeedContainer> SpeedContainerDic = new(); // 速度计算
ConcurrentDictionary<StreamSpec, bool?> Results = new();
var progress = CustomAnsiConsole.Console.Progress().AutoClear(true);
progress.AutoRefresh = DownloaderConfig.MyOptions.LogLevel != LogLevel.OFF;
// 进度条的列定义
var progressColumns = new ProgressColumn[]
{
new TaskDescriptionColumn() { Alignment = Justify.Left },
new RecordingDurationColumn(RecordingDurDic), // 时长显示
new RecordingSizeColumn(RecordingSizeDic), // 大小显示
new RecordingStatusColumn(),
new DownloadSpeedColumn(SpeedContainerDic), // 速度计算
new SpinnerColumn(),
};
if (DownloaderConfig.MyOptions.NoAnsiColor)
{
progressColumns = progressColumns.SkipLast(1).ToArray();
}
progress.Columns(progressColumns);
await progress.StartAsync(async ctx =>
{
// 创建任务
var dic = SelectedSteams.Select(item =>
{
var task = ctx.AddTask(item.ToShortString(), autoStart: false, maxValue: 0);
SpeedContainerDic[task.Id] = new SpeedContainer(); // 速度计算
RecordingDurDic[task.Id] = 0;
RecordingSizeDic[task.Id] = 0;
return (item, task);
}).ToDictionary(item => item.item, item => item.task);
DownloaderConfig.MyOptions.LiveRecordLimit ??= TimeSpan.MaxValue;
var limit = DownloaderConfig.MyOptions.LiveRecordLimit;
if (limit != TimeSpan.MaxValue)
Logger.WarnMarkUp($"[darkorange3_1]{ResString.liveLimit}{GlobalUtil.FormatTime((int)limit.Value.TotalSeconds)}[/]");
// 录制直播时，用户选了几个流就并发录几个
var options = new ParallelOptions()
{
MaxDegreeOfParallelism = SelectedSteams.Count
};
// 并发下载
await Parallel.ForEachAsync(dic, options, async (kp, _) =>
{
var task = kp.Value;
var consumerTask = RecordStreamAsync(kp.Key, task, SpeedContainerDic[task.Id]);
Results[kp.Key] = await consumerTask;
});
});
var success = Results.Values.All(v => v == true);
return success;
}
}

759
src/N_m3u8DL-RE/DownloadManager/SimpleDownloadManager.cs Normal file
View File

@ -0,0 +1,759 @@
using Mp4SubtitleParser;
using N_m3u8DL_RE.Column;
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Config;
using N_m3u8DL_RE.Downloader;
using N_m3u8DL_RE.Entity;
using N_m3u8DL_RE.Parser;
using N_m3u8DL_RE.Parser.Mp4;
using N_m3u8DL_RE.Util;
using Spectre.Console;
using System.Collections.Concurrent;
using System.Text;
using N_m3u8DL_RE.Enum;
namespace N_m3u8DL_RE.DownloadManager;
internal class SimpleDownloadManager
{
IDownloader Downloader;
DownloaderConfig DownloaderConfig;
StreamExtractor StreamExtractor;
List<StreamSpec> SelectedSteams;
List<OutputFile> OutputFiles = [];
public SimpleDownloadManager(DownloaderConfig downloaderConfig, List<StreamSpec> selectedSteams, StreamExtractor streamExtractor)
{
this.DownloaderConfig = downloaderConfig;
this.SelectedSteams = selectedSteams;
this.StreamExtractor = streamExtractor;
Downloader = new SimpleDownloader(DownloaderConfig);
}
// 从文件读取KEY
private async Task SearchKeyAsync(string? currentKID)
{
var _key = await MP4DecryptUtil.SearchKeyFromFileAsync(DownloaderConfig.MyOptions.KeyTextFile, currentKID);
if (_key != null)
{
if (DownloaderConfig.MyOptions.Keys == null)
DownloaderConfig.MyOptions.Keys = [_key];
else
DownloaderConfig.MyOptions.Keys = [..DownloaderConfig.MyOptions.Keys, _key];
}
}
private void ChangeSpecInfo(StreamSpec streamSpec, List<Mediainfo> mediainfos, ref bool useAACFilter)
{
if (!DownloaderConfig.MyOptions.BinaryMerge && mediainfos.Any(m => m.DolbyVison))
{
DownloaderConfig.MyOptions.BinaryMerge = true;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge2}[/]");
}
if (DownloaderConfig.MyOptions.MuxAfterDone && mediainfos.Any(m => m.DolbyVison))
{
DownloaderConfig.MyOptions.MuxAfterDone = false;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge5}[/]");
}
if (mediainfos.Where(m => m.Type == "Audio").All(m => m.BaseInfo!.Contains("aac")))
{
useAACFilter = true;
}
if (mediainfos.All(m => m.Type == "Audio"))
{
streamSpec.MediaType = MediaType.AUDIO;
}
else if (mediainfos.All(m => m.Type == "Subtitle"))
{
streamSpec.MediaType = MediaType.SUBTITLES;
if (streamSpec.Extension is null or "ts")
streamSpec.Extension = "vtt";
}
}
private async Task<bool> DownloadStreamAsync(StreamSpec streamSpec, ProgressTask task, SpeedContainer speedContainer)
{
speedContainer.ResetVars();
bool useAACFilter = false; // ffmpeg合并flag
List<Mediainfo> mediaInfos = [];
ConcurrentDictionary<MediaSegment, DownloadResult?> FileDic = new();
var segments = streamSpec.Playlist?.MediaParts.SelectMany(m => m.MediaSegments);
if (segments == null || !segments.Any()) return false;
// 单分段尝试切片并行下载
if (segments.Count() == 1)
{
var splitSegments = await LargeSingleFileSplitUtil.SplitUrlAsync(segments.First(), DownloaderConfig.Headers);
if (splitSegments != null)
{
segments = splitSegments;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.singleFileSplitWarn}[/]");
if (DownloaderConfig.MyOptions.MP4RealTimeDecryption)
{
DownloaderConfig.MyOptions.MP4RealTimeDecryption = false;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.singleFileRealtimeDecryptWarn}[/]");
}
}
else speedContainer.SingleSegment = true;
}
var type = streamSpec.MediaType ?? Common.Enum.MediaType.VIDEO;
var dirName = $"{task.Id}_{OtherUtil.GetValidFileName(streamSpec.GroupId ?? "", "-")}_{streamSpec.Codecs}_{streamSpec.Bandwidth}_{streamSpec.Language}";
var tmpDir = Path.Combine(DownloaderConfig.DirPrefix, dirName);
var saveDir = DownloaderConfig.MyOptions.SaveDir ?? Environment.CurrentDirectory;
var saveName = DownloaderConfig.MyOptions.SaveName != null ? $"{DownloaderConfig.MyOptions.SaveName}.{streamSpec.Language}".TrimEnd('.') : dirName;
var headers = DownloaderConfig.Headers;
var decryptionBinaryPath = DownloaderConfig.MyOptions.DecryptionBinaryPath!;
var decryptEngine = DownloaderConfig.MyOptions.DecryptionEngine;
var mp4InitFile = "";
var currentKID = "";
var readInfo = false; // 是否读取过
var mp4Info = new ParsedMP4Info();
// 用户自定义范围导致被跳过的时长 计算字幕偏移使用
var skippedDur = streamSpec.SkippedDuration ?? 0d;
Logger.Debug($"dirName: {dirName}; tmpDir: {tmpDir}; saveDir: {saveDir}; saveName: {saveName}");
// 创建文件夹
if (!Directory.Exists(tmpDir)) Directory.CreateDirectory(tmpDir);
if (!Directory.Exists(saveDir)) Directory.CreateDirectory(saveDir);
var totalCount = segments.Count();
if (streamSpec.Playlist?.MediaInit != null)
{
totalCount++;
}
task.MaxValue = totalCount;
task.StartTask();
// 开始下载
Logger.InfoMarkUp(ResString.startDownloading + streamSpec.ToShortString());
// 对于CENC，全部自动开启二进制合并
if (!DownloaderConfig.MyOptions.BinaryMerge && totalCount >= 1 && streamSpec.Playlist!.MediaParts.First().MediaSegments.First().EncryptInfo.Method == Common.Enum.EncryptMethod.CENC)
{
DownloaderConfig.MyOptions.BinaryMerge = true;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge4}[/]");
}
// 下载init
if (streamSpec.Playlist?.MediaInit != null)
{
// 对于fMP4，自动开启二进制合并
if (!DownloaderConfig.MyOptions.BinaryMerge && streamSpec.MediaType != MediaType.SUBTITLES)
{
DownloaderConfig.MyOptions.BinaryMerge = true;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge}[/]");
}
var path = Path.Combine(tmpDir, "_init.mp4.tmp");
var result = await Downloader.DownloadSegmentAsync(streamSpec.Playlist.MediaInit, path, speedContainer, headers);
FileDic[streamSpec.Playlist.MediaInit] = result;
if (result is not { Success: true })
{
throw new Exception("Download init file failed!");
}
mp4InitFile = result.ActualFilePath;
task.Increment(1);
// 读取mp4信息
if (result is { Success: true })
{
mp4Info = MP4DecryptUtil.GetMP4Info(result.ActualFilePath);
currentKID = mp4Info.KID;
// try shaka packager, which can handle WebM
if (string.IsNullOrEmpty(currentKID) && DownloaderConfig.MyOptions.DecryptionEngine == DecryptEngine.SHAKA_PACKAGER) {
currentKID = MP4DecryptUtil.ReadInitShaka(result.ActualFilePath, decryptionBinaryPath);
}
// 从文件读取KEY
await SearchKeyAsync(currentKID);
// 实时解密
if ((streamSpec.Playlist.MediaInit.IsEncrypted || !string.IsNullOrEmpty(currentKID)) && DownloaderConfig.MyOptions.MP4RealTimeDecryption && !string.IsNullOrEmpty(currentKID) && StreamExtractor.ExtractorType != ExtractorType.MSS)
{
var enc = result.ActualFilePath;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
var dResult = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID, isMultiDRM: mp4Info.isMultiDRM);
if (dResult)
{
FileDic[streamSpec.Playlist.MediaInit]!.ActualFilePath = dec;
}
}
// ffmpeg读取信息
if (!readInfo)
{
Logger.WarnMarkUp(ResString.readingInfo);
mediaInfos = await MediainfoUtil.ReadInfoAsync(DownloaderConfig.MyOptions.FFmpegBinaryPath!, result.ActualFilePath);
mediaInfos.ForEach(info => Logger.InfoMarkUp(info.ToStringMarkUp()));
ChangeSpecInfo(streamSpec, mediaInfos, ref useAACFilter);
readInfo = true;
}
}
}
// 计算填零个数
var pad = "0".PadLeft(segments.Count().ToString().Length, '0');
// 下载第一个分片
if (!readInfo || StreamExtractor.ExtractorType == ExtractorType.MSS)
{
var seg = segments.First();
segments = segments.Skip(1);
var index = seg.Index;
var path = Path.Combine(tmpDir, index.ToString(pad) + $".{streamSpec.Extension ?? "clip"}.tmp");
var result = await Downloader.DownloadSegmentAsync(seg, path, speedContainer, headers);
FileDic[seg] = result;
if (result is not { Success: true })
{
throw new Exception("Download first segment failed!");
}
task.Increment(1);
if (result is { Success: true })
{
// 修复MSS init
if (StreamExtractor.ExtractorType == ExtractorType.MSS)
{
var processor = new MSSMoovProcessor(streamSpec);
var header = processor.GenHeader(File.ReadAllBytes(result.ActualFilePath));
await File.WriteAllBytesAsync(FileDic[streamSpec.Playlist!.MediaInit!]!.ActualFilePath, header);
if (seg.IsEncrypted && DownloaderConfig.MyOptions.MP4RealTimeDecryption && !string.IsNullOrEmpty(currentKID))
{
// 需要重新解密init
var enc = FileDic[streamSpec.Playlist!.MediaInit!]!.ActualFilePath;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
var dResult = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID);
if (dResult)
{
FileDic[streamSpec.Playlist!.MediaInit!]!.ActualFilePath = dec;
}
}
}
// 读取init信息
if (string.IsNullOrEmpty(currentKID))
{
currentKID = MP4DecryptUtil.GetMP4Info(result.ActualFilePath).KID;
}
// try shaka packager, which can handle WebM
if (string.IsNullOrEmpty(currentKID) && DownloaderConfig.MyOptions.DecryptionEngine == DecryptEngine.SHAKA_PACKAGER) {
currentKID = MP4DecryptUtil.ReadInitShaka(result.ActualFilePath, decryptionBinaryPath);
}
// 从文件读取KEY
await SearchKeyAsync(currentKID);
// 实时解密
if (seg.IsEncrypted && DownloaderConfig.MyOptions.MP4RealTimeDecryption && !string.IsNullOrEmpty(currentKID))
{
var enc = result.ActualFilePath;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
mp4Info = MP4DecryptUtil.GetMP4Info(enc);
var dResult = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID, mp4InitFile, isMultiDRM: mp4Info.isMultiDRM);
if (dResult)
{
File.Delete(enc);
result.ActualFilePath = dec;
}
}
if (!readInfo)
{
// ffmpeg读取信息
Logger.WarnMarkUp(ResString.readingInfo);
mediaInfos = await MediainfoUtil.ReadInfoAsync(DownloaderConfig.MyOptions.FFmpegBinaryPath!, result!.ActualFilePath);
mediaInfos.ForEach(info => Logger.InfoMarkUp(info.ToStringMarkUp()));
ChangeSpecInfo(streamSpec, mediaInfos, ref useAACFilter);
readInfo = true;
}
}
}
// 开始下载
var options = new ParallelOptions()
{
MaxDegreeOfParallelism = DownloaderConfig.MyOptions.ThreadCount
};
await Parallel.ForEachAsync(segments, options, async (seg, _) =>
{
var index = seg.Index;
var path = Path.Combine(tmpDir, index.ToString(pad) + $".{streamSpec.Extension ?? "clip"}.tmp");
var result = await Downloader.DownloadSegmentAsync(seg, path, speedContainer, headers);
FileDic[seg] = result;
if (result is { Success: true })
task.Increment(1);
// 实时解密
if (seg.IsEncrypted && DownloaderConfig.MyOptions.MP4RealTimeDecryption && result is { Success: true } && !string.IsNullOrEmpty(currentKID))
{
var enc = result.ActualFilePath;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
mp4Info = MP4DecryptUtil.GetMP4Info(enc);
var dResult = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID, mp4InitFile, isMultiDRM: mp4Info.isMultiDRM);
if (dResult)
{
File.Delete(enc);
result.ActualFilePath = dec;
}
}
});
// 修改输出后缀
var outputExt = "." + streamSpec.Extension;
if (streamSpec.Extension == null) outputExt = ".ts";
else if (streamSpec is { MediaType: MediaType.AUDIO, Extension: "m4s" or "mp4" }) outputExt = ".m4a";
else if (streamSpec.MediaType != MediaType.SUBTITLES && streamSpec.Extension is "m4s" or "mp4") outputExt = ".mp4";
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec.MediaType == MediaType.SUBTITLES)
{
outputExt = DownloaderConfig.MyOptions.SubtitleFormat == Enum.SubtitleFormat.SRT ? ".srt" : ".vtt";
}
var output = Path.Combine(saveDir, saveName + outputExt);
// 检测目标文件是否存在
while (File.Exists(output))
{
Logger.WarnMarkUp($"{Path.GetFileName(output)} => {Path.GetFileName(output = Path.ChangeExtension(output, $"copy" + Path.GetExtension(output)))}");
}
if (!string.IsNullOrEmpty(currentKID) && DownloaderConfig.MyOptions is { MP4RealTimeDecryption: true, Keys.Length: > 0 } && mp4InitFile != "")
{
File.Delete(mp4InitFile);
// shaka/ffmpeg实时解密不需要init文件用于合并
if (decryptEngine != DecryptEngine.MP4DECRYPT)
{
FileDic!.Remove(streamSpec.Playlist!.MediaInit, out _);
}
}
// 校验分片数量
if (DownloaderConfig.MyOptions.CheckSegmentsCount && FileDic.Values.Any(s => s == null))
{
Logger.ErrorMarkUp(ResString.segmentCountCheckNotPass, totalCount, FileDic.Values.Count(s => s != null));
return false;
}
// 移除无效片段
var badKeys = FileDic.Where(i => i.Value == null).Select(i => i.Key);
foreach (var badKey in badKeys)
{
FileDic!.Remove(badKey, out _);
}
// 校验完整性
if (DownloaderConfig.CheckContentLength && FileDic.Values.Any(a => a!.Success == false))
{
return false;
}
// 自动修复VTT raw字幕
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec is { MediaType: Common.Enum.MediaType.SUBTITLES, Extension: not null } && streamSpec.Extension.Contains("vtt"))
{
Logger.WarnMarkUp(ResString.fixingVTT);
// 排序字幕并修正时间戳
bool first = true;
var finalVtt = new WebVttSub();
var keys = FileDic.Keys.OrderBy(k => k.Index);
foreach (var seg in keys)
{
var vttContent = File.ReadAllText(FileDic[seg]!.ActualFilePath);
var vtt = WebVttSub.Parse(vttContent);
// 手动计算MPEGTS
if (finalVtt.MpegtsTimestamp == 0 && vtt.MpegtsTimestamp == 0)
{
vtt.MpegtsTimestamp = 90000 * (long)(skippedDur + keys.Where(s => s.Index < seg.Index).Sum(s => s.Duration));
}
if (first) { finalVtt = vtt; first = false; }
else finalVtt.AddCuesFromOne(vtt);
}
// 写出字幕
var files = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Value).Select(v => v!.ActualFilePath).ToArray();
foreach (var item in files) File.Delete(item);
FileDic.Clear();
var index = 0;
var path = Path.Combine(tmpDir, index.ToString(pad) + ".fix.vtt");
// 设置字幕偏移
finalVtt.LeftShiftTime(TimeSpan.FromSeconds(skippedDur));
var subContentFixed = finalVtt.ToVtt();
// 转换字幕格式
if (DownloaderConfig.MyOptions.SubtitleFormat != Enum.SubtitleFormat.VTT)
{
path = Path.ChangeExtension(path, ".srt");
subContentFixed = finalVtt.ToSrt();
}
await File.WriteAllTextAsync(path, subContentFixed, Encoding.UTF8);
FileDic[keys.First()] = new DownloadResult()
{
ActualContentLength = subContentFixed.Length,
ActualFilePath = path
};
}
// 自动修复VTT mp4字幕
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec.MediaType == Common.Enum.MediaType.SUBTITLES
&& streamSpec.Codecs != "stpp" && streamSpec.Extension != null && streamSpec.Extension.Contains("m4s"))
{
var initFile = FileDic.Values.FirstOrDefault(v => Path.GetFileName(v!.ActualFilePath).StartsWith("_init"));
var iniFileBytes = File.ReadAllBytes(initFile!.ActualFilePath);
var (sawVtt, timescale) = MP4VttUtil.CheckInit(iniFileBytes);
if (sawVtt)
{
Logger.WarnMarkUp(ResString.fixingVTTmp4);
var mp4s = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Value).Select(v => v!.ActualFilePath).Where(p => p.EndsWith(".m4s")).ToArray();
var finalVtt = MP4VttUtil.ExtractSub(mp4s, timescale);
// 写出字幕
var firstKey = FileDic.Keys.First();
var files = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Value).Select(v => v!.ActualFilePath).ToArray();
foreach (var item in files) File.Delete(item);
FileDic.Clear();
var index = 0;
var path = Path.Combine(tmpDir, index.ToString(pad) + ".fix.vtt");
// 设置字幕偏移
finalVtt.LeftShiftTime(TimeSpan.FromSeconds(skippedDur));
var subContentFixed = finalVtt.ToVtt();
// 转换字幕格式
if (DownloaderConfig.MyOptions.SubtitleFormat != Enum.SubtitleFormat.VTT)
{
path = Path.ChangeExtension(path, ".srt");
subContentFixed = finalVtt.ToSrt();
}
await File.WriteAllTextAsync(path, subContentFixed, Encoding.UTF8);
FileDic[firstKey] = new DownloadResult()
{
ActualContentLength = subContentFixed.Length,
ActualFilePath = path
};
}
}
// 自动修复TTML raw字幕
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec is { MediaType: Common.Enum.MediaType.SUBTITLES, Extension: not null } && streamSpec.Extension.Contains("ttml"))
{
Logger.WarnMarkUp(ResString.fixingTTML);
var first = true;
var finalVtt = new WebVttSub();
var keys = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Key);
foreach (var seg in keys)
{
var vtt = MP4TtmlUtil.ExtractFromTTML(FileDic[seg]!.ActualFilePath, 0);
// 手动计算MPEGTS
if (finalVtt.MpegtsTimestamp == 0 && vtt.MpegtsTimestamp == 0)
{
vtt.MpegtsTimestamp = 90000 * (long)(skippedDur + keys.Where(s => s.Index < seg.Index).Sum(s => s.Duration));
}
if (first) { finalVtt = vtt; first = false; }
else finalVtt.AddCuesFromOne(vtt);
}
// 写出字幕
var firstKey = FileDic.Keys.First();
var files = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Value).Select(v => v!.ActualFilePath).ToArray();
// 处理图形字幕
await SubtitleUtil.TryWriteImagePngsAsync(finalVtt, tmpDir);
foreach (var item in files) File.Delete(item);
FileDic.Clear();
var index = 0;
var path = Path.Combine(tmpDir, index.ToString(pad) + ".fix.vtt");
// 设置字幕偏移
finalVtt.LeftShiftTime(TimeSpan.FromSeconds(skippedDur));
var subContentFixed = finalVtt.ToVtt();
// 转换字幕格式
if (DownloaderConfig.MyOptions.SubtitleFormat != Enum.SubtitleFormat.VTT)
{
path = Path.ChangeExtension(path, ".srt");
subContentFixed = finalVtt.ToSrt();
}
await File.WriteAllTextAsync(path, subContentFixed, Encoding.UTF8);
FileDic[firstKey] = new DownloadResult()
{
ActualContentLength = subContentFixed.Length,
ActualFilePath = path
};
}
// 自动修复TTML mp4字幕
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec is { MediaType: Common.Enum.MediaType.SUBTITLES, Extension: not null } && streamSpec.Extension.Contains("m4s")
&& streamSpec.Codecs != null && streamSpec.Codecs.Contains("stpp"))
{
Logger.WarnMarkUp(ResString.fixingTTMLmp4);
// sawTtml暂时不判断
// var initFile = FileDic.Values.Where(v => Path.GetFileName(v!.ActualFilePath).StartsWith("_init")).FirstOrDefault();
// var iniFileBytes = File.ReadAllBytes(initFile!.ActualFilePath);
// var sawTtml = MP4TtmlUtil.CheckInit(iniFileBytes);
var first = true;
var finalVtt = new WebVttSub();
var keys = FileDic.OrderBy(s => s.Key.Index).Where(v => v.Value!.ActualFilePath.EndsWith(".m4s")).Select(s => s.Key);
foreach (var seg in keys)
{
var vtt = MP4TtmlUtil.ExtractFromMp4(FileDic[seg]!.ActualFilePath, 0);
// 手动计算MPEGTS
if (finalVtt.MpegtsTimestamp == 0 && vtt.MpegtsTimestamp == 0)
{
vtt.MpegtsTimestamp = 90000 * (long)(skippedDur + keys.Where(s => s.Index < seg.Index).Sum(s => s.Duration));
}
if (first) { finalVtt = vtt; first = false; }
else finalVtt.AddCuesFromOne(vtt);
}
// 写出字幕
var firstKey = FileDic.Keys.First();
var files = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Value).Select(v => v!.ActualFilePath).ToArray();
// 处理图形字幕
await SubtitleUtil.TryWriteImagePngsAsync(finalVtt, tmpDir);
foreach (var item in files) File.Delete(item);
FileDic.Clear();
var index = 0;
var path = Path.Combine(tmpDir, index.ToString(pad) + ".fix.vtt");
// 设置字幕偏移
finalVtt.LeftShiftTime(TimeSpan.FromSeconds(skippedDur));
var subContentFixed = finalVtt.ToVtt();
// 转换字幕格式
if (DownloaderConfig.MyOptions.SubtitleFormat != Enum.SubtitleFormat.VTT)
{
path = Path.ChangeExtension(path, ".srt");
subContentFixed = finalVtt.ToSrt();
}
await File.WriteAllTextAsync(path, subContentFixed, Encoding.UTF8);
FileDic[firstKey] = new DownloadResult()
{
ActualContentLength = subContentFixed.Length,
ActualFilePath = path
};
}
bool mergeSuccess = false;
// 合并
if (!DownloaderConfig.MyOptions.SkipMerge)
{
// 字幕也使用二进制合并
if (DownloaderConfig.MyOptions.BinaryMerge || streamSpec.MediaType == MediaType.SUBTITLES)
{
Logger.InfoMarkUp(ResString.binaryMerge);
var files = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Value).Select(v => v!.ActualFilePath).ToArray();
MergeUtil.CombineMultipleFilesIntoSingleFile(files, output);
mergeSuccess = true;
}
else
{
// ffmpeg合并
var files = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Value).Select(v => v!.ActualFilePath).ToArray();
Logger.InfoMarkUp(ResString.ffmpegMerge);
var ext = streamSpec.MediaType == MediaType.AUDIO ? "m4a" : "mp4";
var ffOut = Path.Combine(Path.GetDirectoryName(output)!, Path.GetFileNameWithoutExtension(output) + $".{ext}");
// 检测目标文件是否存在
while (File.Exists(ffOut))
{
Logger.WarnMarkUp($"{Path.GetFileName(ffOut)} => {Path.GetFileName(ffOut = Path.ChangeExtension(ffOut, $"copy" + Path.GetExtension(ffOut)))}");
}
// 大于1800分片，需要分步骤合并
if (files.Length >= 1800)
{
Logger.WarnMarkUp(ResString.partMerge);
files = MergeUtil.PartialCombineMultipleFiles(files);
FileDic.Clear();
foreach (var item in files)
{
FileDic[new MediaSegment() { Url = item }] = new DownloadResult()
{
ActualFilePath = item
};
}
}
mergeSuccess = MergeUtil.MergeByFFmpeg(DownloaderConfig.MyOptions.FFmpegBinaryPath!, files, Path.ChangeExtension(ffOut, null), ext, useAACFilter, writeDate: !DownloaderConfig.MyOptions.NoDateInfo, useConcatDemuxer: DownloaderConfig.MyOptions.UseFFmpegConcatDemuxer);
if (mergeSuccess) output = ffOut;
}
}
// 删除临时文件夹
if (DownloaderConfig.MyOptions is { SkipMerge: false, DelAfterDone: true } && mergeSuccess)
{
var files = FileDic.Values.Select(v => v!.ActualFilePath);
foreach (var file in files)
{
File.Delete(file);
}
OtherUtil.SafeDeleteDir(tmpDir);
}
// 重新读取init信息
if (mergeSuccess && totalCount >= 1 && string.IsNullOrEmpty(currentKID) && streamSpec.Playlist!.MediaParts.First().MediaSegments.First().EncryptInfo.Method != Common.Enum.EncryptMethod.NONE)
{
currentKID = MP4DecryptUtil.GetMP4Info(output).KID;
// try shaka packager, which can handle WebM
if (string.IsNullOrEmpty(currentKID) && DownloaderConfig.MyOptions.DecryptionEngine == DecryptEngine.SHAKA_PACKAGER) {
currentKID = MP4DecryptUtil.ReadInitShaka(output, decryptionBinaryPath);
}
// 从文件读取KEY
await SearchKeyAsync(currentKID);
}
// 调用mp4decrypt解密
if (mergeSuccess && File.Exists(output) && !string.IsNullOrEmpty(currentKID) && DownloaderConfig.MyOptions is { MP4RealTimeDecryption: false, Keys.Length: > 0 })
{
var enc = output;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
mp4Info = MP4DecryptUtil.GetMP4Info(enc);
Logger.InfoMarkUp($"[grey]Decrypting using {decryptEngine}...[/]");
var result = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID, isMultiDRM: mp4Info.isMultiDRM);
if (result)
{
File.Delete(enc);
File.Move(dec, enc);
}
}
// 记录所有文件信息
if (File.Exists(output))
{
OutputFiles.Add(new OutputFile()
{
Index = task.Id,
FilePath = output,
LangCode = streamSpec.Language,
Description = streamSpec.Name,
Mediainfos = mediaInfos,
MediaType = streamSpec.MediaType,
});
}
return true;
}
public async Task<bool> StartDownloadAsync()
{
ConcurrentDictionary<int, SpeedContainer> SpeedContainerDic = new(); // 速度计算
ConcurrentDictionary<StreamSpec, bool?> Results = new();
var progress = CustomAnsiConsole.Console.Progress().AutoClear(true);
progress.AutoRefresh = DownloaderConfig.MyOptions.LogLevel != LogLevel.OFF;
// 进度条的列定义
var progressColumns = new ProgressColumn[]
{
new TaskDescriptionColumn() { Alignment = Justify.Left },
new ProgressBarColumn(){ Width = 30 },
new MyPercentageColumn(),
new DownloadStatusColumn(SpeedContainerDic),
new DownloadSpeedColumn(SpeedContainerDic), // 速度计算
new RemainingTimeColumn(),
new SpinnerColumn(),
};
if (DownloaderConfig.MyOptions.NoAnsiColor)
{
progressColumns = progressColumns.SkipLast(1).ToArray();
}
progress.Columns(progressColumns);
if (DownloaderConfig.MyOptions is { MP4RealTimeDecryption: true, DecryptionEngine: not DecryptEngine.SHAKA_PACKAGER, Keys.Length: > 0 })
Logger.WarnMarkUp($"[darkorange3_1]{ResString.realTimeDecMessage}[/]");
await progress.StartAsync(async ctx =>
{
// 创建任务
var dic = SelectedSteams.Select(item =>
{
var description = item.ToShortShortString();
var task = ctx.AddTask(description, autoStart: false);
SpeedContainerDic[task.Id] = new SpeedContainer(); // 速度计算
// 限速设置
if (DownloaderConfig.MyOptions.MaxSpeed != null)
{
SpeedContainerDic[task.Id].SpeedLimit = DownloaderConfig.MyOptions.MaxSpeed.Value;
}
return (item, task);
}).ToDictionary(item => item.item, item => item.task);
if (!DownloaderConfig.MyOptions.ConcurrentDownload)
{
// 遍历，顺序下载
foreach (var kp in dic)
{
var task = kp.Value;
var result = await DownloadStreamAsync(kp.Key, task, SpeedContainerDic[task.Id]);
Results[kp.Key] = result;
// 失败不再下载后续
if (!result) break;
}
}
else
{
// 并发下载
await Parallel.ForEachAsync(dic, async (kp, _) =>
{
var task = kp.Value;
var result = await DownloadStreamAsync(kp.Key, task, SpeedContainerDic[task.Id]);
Results[kp.Key] = result;
});
}
});
var success = Results.Values.All(v => v == true);
// 删除临时文件夹
if (DownloaderConfig.MyOptions is { SkipMerge: false, DelAfterDone: true } && success)
{
foreach (var item in StreamExtractor.RawFiles)
{
var file = Path.Combine(DownloaderConfig.DirPrefix, item.Key);
if (File.Exists(file)) File.Delete(file);
}
OtherUtil.SafeDeleteDir(DownloaderConfig.DirPrefix);
}
// 混流
if (success && DownloaderConfig.MyOptions.MuxAfterDone && OutputFiles.Count > 0)
{
OutputFiles = OutputFiles.OrderBy(o => o.Index).ToList();
// 是否跳过字幕
if (DownloaderConfig.MyOptions.MuxOptions!.SkipSubtitle)
{
OutputFiles = OutputFiles.Where(o => o.MediaType != MediaType.SUBTITLES).ToList();
}
if (DownloaderConfig.MyOptions.MuxImports != null)
{
OutputFiles.AddRange(DownloaderConfig.MyOptions.MuxImports);
}
OutputFiles.ForEach(f => Logger.WarnMarkUp($"[grey]{Path.GetFileName(f.FilePath).EscapeMarkup()}[/]"));
var saveDir = DownloaderConfig.MyOptions.SaveDir ?? Environment.CurrentDirectory;
var ext = OtherUtil.GetMuxExtension(DownloaderConfig.MyOptions.MuxOptions.MuxFormat);
var dirName = Path.GetFileName(DownloaderConfig.DirPrefix);
var outName = $"{dirName}.MUX";
var outPath = Path.Combine(saveDir, outName);
Logger.WarnMarkUp($"Muxing to [grey]{outName.EscapeMarkup()}{ext}[/]");
var result = false;
if (DownloaderConfig.MyOptions.MuxOptions.UseMkvmerge) result = MergeUtil.MuxInputsByMkvmerge(DownloaderConfig.MyOptions.MkvmergeBinaryPath!, OutputFiles.ToArray(), outPath);
else result = MergeUtil.MuxInputsByFFmpeg(DownloaderConfig.MyOptions.FFmpegBinaryPath!, OutputFiles.ToArray(), outPath, DownloaderConfig.MyOptions.MuxOptions.MuxFormat, !DownloaderConfig.MyOptions.NoDateInfo);
// 完成后删除各轨道文件
if (result)
{
if (!DownloaderConfig.MyOptions.MuxOptions.KeepFiles)
{
Logger.WarnMarkUp("[grey]Cleaning files...[/]");
OutputFiles.ForEach(f => File.Delete(f.FilePath));
var tmpDir = DownloaderConfig.MyOptions.TmpDir ?? Environment.CurrentDirectory;
OtherUtil.SafeDeleteDir(tmpDir);
}
}
else
{
success = false;
Logger.ErrorMarkUp($"Mux failed");
}
// 判断是否要改名
var newPath = Path.ChangeExtension(outPath, ext);
if (result && !File.Exists(newPath))
{
Logger.WarnMarkUp($"Rename to [grey]{Path.GetFileName(newPath).EscapeMarkup()}[/]");
File.Move(outPath + ext, newPath);
}
}
return success;
}
}

921
src/N_m3u8DL-RE/DownloadManager/SimpleLiveRecordManager2.cs Normal file
View File

@ -0,0 +1,921 @@
using Mp4SubtitleParser;
using N_m3u8DL_RE.Column;
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Config;
using N_m3u8DL_RE.Downloader;
using N_m3u8DL_RE.Entity;
using N_m3u8DL_RE.Parser;
using N_m3u8DL_RE.Parser.Mp4;
using N_m3u8DL_RE.Util;
using Spectre.Console;
using System.Collections.Concurrent;
using System.IO.Pipes;
using System.Text;
using System.Threading.Tasks.Dataflow;
using N_m3u8DL_RE.Enum;
namespace N_m3u8DL_RE.DownloadManager;
internal class SimpleLiveRecordManager2
{
IDownloader Downloader;
DownloaderConfig DownloaderConfig;
StreamExtractor StreamExtractor;
List<StreamSpec> SelectedSteams;
ConcurrentDictionary<int, string> PipeSteamNamesDic = new();
List<OutputFile> OutputFiles = [];
DateTime? PublishDateTime;
bool STOP_FLAG = false;
int WAIT_SEC = 0; // 刷新间隔
ConcurrentDictionary<int, int> RecordedDurDic = new(); // 已录制时长
ConcurrentDictionary<int, int> RefreshedDurDic = new(); // 已刷新出的时长
ConcurrentDictionary<int, BufferBlock<List<MediaSegment>>> BlockDic = new(); // 各流的Block
ConcurrentDictionary<int, bool> SamePathDic = new(); // 各流是否allSamePath
ConcurrentDictionary<int, bool> RecordLimitReachedDic = new(); // 各流是否达到上限
ConcurrentDictionary<int, string> LastFileNameDic = new(); // 上次下载的文件名
ConcurrentDictionary<int, long> MaxIndexDic = new(); // 最大Index
ConcurrentDictionary<int, long> DateTimeDic = new(); // 上次下载的dateTime
CancellationTokenSource CancellationTokenSource = new(); // 取消Wait
private readonly Lock lockObj = new();
TimeSpan? audioStart = null;
public SimpleLiveRecordManager2(DownloaderConfig downloaderConfig, List<StreamSpec> selectedSteams, StreamExtractor streamExtractor)
{
this.DownloaderConfig = downloaderConfig;
Downloader = new SimpleDownloader(DownloaderConfig);
PublishDateTime = selectedSteams.FirstOrDefault()?.PublishTime;
StreamExtractor = streamExtractor;
SelectedSteams = selectedSteams;
}
// 从文件读取KEY
private async Task SearchKeyAsync(string? currentKID)
{
var _key = await MP4DecryptUtil.SearchKeyFromFileAsync(DownloaderConfig.MyOptions.KeyTextFile, currentKID);
if (_key != null)
{
if (DownloaderConfig.MyOptions.Keys == null)
DownloaderConfig.MyOptions.Keys = [_key];
else
DownloaderConfig.MyOptions.Keys = [..DownloaderConfig.MyOptions.Keys, _key];
}
}
/// <summary>
/// 获取时间戳
/// </summary>
/// <param name="dateTime"></param>
/// <returns></returns>
private long GetUnixTimestamp(DateTime dateTime)
{
return new DateTimeOffset(dateTime.ToUniversalTime()).ToUnixTimeSeconds();
}
/// <summary>
/// 获取分段文件夹
/// </summary>
/// <param name="segment"></param>
/// <param name="allHasDatetime"></param>
/// <returns></returns>
private string GetSegmentName(MediaSegment segment, bool allHasDatetime, bool allSamePath)
{
if (!string.IsNullOrEmpty(segment.NameFromVar))
{
return segment.NameFromVar;
}
bool hls = StreamExtractor.ExtractorType == ExtractorType.HLS;
string name = OtherUtil.GetFileNameFromInput(segment.Url, false);
if (allSamePath)
{
name = OtherUtil.GetValidFileName(segment.Url.Split('?').Last(), "_");
}
if (hls && allHasDatetime)
{
name = GetUnixTimestamp(segment.DateTime!.Value).ToString();
}
else if (hls)
{
name = segment.Index.ToString();
}
return name;
}
private void ChangeSpecInfo(StreamSpec streamSpec, List<Mediainfo> mediainfos, ref bool useAACFilter)
{
if (!DownloaderConfig.MyOptions.BinaryMerge && mediainfos.Any(m => m.DolbyVison))
{
DownloaderConfig.MyOptions.BinaryMerge = true;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge2}[/]");
}
if (DownloaderConfig.MyOptions.MuxAfterDone && mediainfos.Any(m => m.DolbyVison))
{
DownloaderConfig.MyOptions.MuxAfterDone = false;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge5}[/]");
}
if (mediainfos.Where(m => m.Type == "Audio").All(m => m.BaseInfo!.Contains("aac")))
{
useAACFilter = true;
}
if (mediainfos.All(m => m.Type == "Audio") && streamSpec.MediaType != MediaType.AUDIO)
{
streamSpec.MediaType = MediaType.AUDIO;
}
else if (mediainfos.All(m => m.Type == "Subtitle") && streamSpec.MediaType != MediaType.SUBTITLES)
{
streamSpec.MediaType = MediaType.SUBTITLES;
if (streamSpec.Extension is null or "ts")
streamSpec.Extension = "vtt";
}
}
private async Task<bool> RecordStreamAsync(StreamSpec streamSpec, ProgressTask task, SpeedContainer speedContainer, BufferBlock<List<MediaSegment>> source)
{
var baseTimestamp = PublishDateTime == null ? 0L : (long)(PublishDateTime.Value.ToUniversalTime() - new DateTime(1970, 1, 1, 0, 0, 0, 0)).TotalMilliseconds;
var decryptionBinaryPath = DownloaderConfig.MyOptions.DecryptionBinaryPath!;
var mp4InitFile = "";
var currentKID = "";
var readInfo = false; // 是否读取过
bool useAACFilter = false; // ffmpeg合并flag
bool initDownloaded = false; // 是否下载过init文件
ConcurrentDictionary<MediaSegment, DownloadResult?> FileDic = new();
List<Mediainfo> mediaInfos = [];
Stream? fileOutputStream = null;
WebVttSub currentVtt = new(); // 字幕流始终维护一个实例
bool firstSub = true;
task.StartTask();
var name = streamSpec.ToShortString();
var type = streamSpec.MediaType ?? Common.Enum.MediaType.VIDEO;
var dirName = $"{task.Id}_{OtherUtil.GetValidFileName(streamSpec.GroupId ?? "", "-")}_{streamSpec.Codecs}_{streamSpec.Bandwidth}_{streamSpec.Language}";
var tmpDir = Path.Combine(DownloaderConfig.DirPrefix, dirName);
var saveDir = DownloaderConfig.MyOptions.SaveDir ?? Environment.CurrentDirectory;
var saveName = DownloaderConfig.MyOptions.SaveName != null ? $"{DownloaderConfig.MyOptions.SaveName}.{streamSpec.Language}".TrimEnd('.') : dirName;
var headers = DownloaderConfig.Headers;
var decryptEngine = DownloaderConfig.MyOptions.DecryptionEngine;
Logger.Debug($"dirName: {dirName}; tmpDir: {tmpDir}; saveDir: {saveDir}; saveName: {saveName}");
// 创建文件夹
if (!Directory.Exists(tmpDir)) Directory.CreateDirectory(tmpDir);
if (!Directory.Exists(saveDir)) Directory.CreateDirectory(saveDir);
while (true && await source.OutputAvailableAsync())
{
// 接收新片段 且总是拿全部未处理的片段
// 有时每次只有很少的片段，但是之前的片段下载慢，导致后面还没下载的片段都失效了
// TryReceiveAll可以稍微缓解一下
source.TryReceiveAll(out IList<List<MediaSegment>>? segmentsList);
var segments = segmentsList!.SelectMany(s => s);
if (segments == null || !segments.Any()) continue;
var segmentsDuration = segments.Sum(s => s.Duration);
Logger.DebugMarkUp(string.Join(",", segments.Select(sss => GetSegmentName(sss, false, false))));
// 下载init
if (!initDownloaded && streamSpec.Playlist?.MediaInit != null)
{
task.MaxValue += 1;
// 对于fMP4，自动开启二进制合并
if (!DownloaderConfig.MyOptions.BinaryMerge && streamSpec.MediaType != MediaType.SUBTITLES)
{
DownloaderConfig.MyOptions.BinaryMerge = true;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge}[/]");
}
var path = Path.Combine(tmpDir, "_init.mp4.tmp");
var result = await Downloader.DownloadSegmentAsync(streamSpec.Playlist.MediaInit, path, speedContainer, headers);
FileDic[streamSpec.Playlist.MediaInit] = result;
if (result is not { Success: true })
{
throw new Exception("Download init file failed!");
}
mp4InitFile = result.ActualFilePath;
task.Increment(1);
// 读取mp4信息
if (result is { Success: true })
{
currentKID = MP4DecryptUtil.GetMP4Info(result.ActualFilePath).KID;
// 从文件读取KEY
await SearchKeyAsync(currentKID);
// 实时解密
if ((streamSpec.Playlist.MediaInit.IsEncrypted || !string.IsNullOrEmpty(currentKID)) && DownloaderConfig.MyOptions.MP4RealTimeDecryption && !string.IsNullOrEmpty(currentKID) && StreamExtractor.ExtractorType != ExtractorType.MSS)
{
var enc = result.ActualFilePath;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
var dResult = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID);
if (dResult)
{
FileDic[streamSpec.Playlist.MediaInit]!.ActualFilePath = dec;
}
}
// ffmpeg读取信息
if (!readInfo)
{
Logger.WarnMarkUp(ResString.readingInfo);
mediaInfos = await MediainfoUtil.ReadInfoAsync(DownloaderConfig.MyOptions.FFmpegBinaryPath!, result.ActualFilePath);
mediaInfos.ForEach(info => Logger.InfoMarkUp(info.ToStringMarkUp()));
lock (lockObj)
{
if (audioStart == null) audioStart = mediaInfos.FirstOrDefault(x => x.Type == "Audio")?.StartTime;
}
ChangeSpecInfo(streamSpec, mediaInfos, ref useAACFilter);
readInfo = true;
}
initDownloaded = true;
}
}
var allHasDatetime = segments.All(s => s.DateTime != null);
if (!SamePathDic.ContainsKey(task.Id))
{
var allName = segments.Select(s => OtherUtil.GetFileNameFromInput(s.Url, false));
var allSamePath = allName.Count() > 1 && allName.Distinct().Count() == 1;
SamePathDic[task.Id] = allSamePath;
}
// 下载第一个分片
if (!readInfo || StreamExtractor.ExtractorType == ExtractorType.MSS)
{
var seg = segments.First();
segments = segments.Skip(1);
// 获取文件名
var filename = GetSegmentName(seg, allHasDatetime, SamePathDic[task.Id]);
var index = seg.Index;
var path = Path.Combine(tmpDir, filename + $".{streamSpec.Extension ?? "clip"}.tmp");
var result = await Downloader.DownloadSegmentAsync(seg, path, speedContainer, headers);
FileDic[seg] = result;
if (result is not { Success: true })
{
throw new Exception("Download first segment failed!");
}
task.Increment(1);
if (result is { Success: true })
{
// 修复MSS init
if (StreamExtractor.ExtractorType == ExtractorType.MSS)
{
var processor = new MSSMoovProcessor(streamSpec);
var header = processor.GenHeader(File.ReadAllBytes(result.ActualFilePath));
await File.WriteAllBytesAsync(FileDic[streamSpec.Playlist!.MediaInit!]!.ActualFilePath, header);
if (seg.IsEncrypted && DownloaderConfig.MyOptions.MP4RealTimeDecryption && !string.IsNullOrEmpty(currentKID))
{
// 需要重新解密init
var enc = FileDic[streamSpec.Playlist!.MediaInit!]!.ActualFilePath;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
var dResult = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID);
if (dResult)
{
FileDic[streamSpec.Playlist!.MediaInit!]!.ActualFilePath = dec;
}
}
}
// 读取init信息
if (string.IsNullOrEmpty(currentKID))
{
currentKID = MP4DecryptUtil.GetMP4Info(result.ActualFilePath).KID;
}
// 从文件读取KEY
await SearchKeyAsync(currentKID);
// 实时解密
if (seg.IsEncrypted && DownloaderConfig.MyOptions.MP4RealTimeDecryption && !string.IsNullOrEmpty(currentKID))
{
var enc = result.ActualFilePath;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
var dResult = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID, mp4InitFile);
if (dResult)
{
File.Delete(enc);
result.ActualFilePath = dec;
}
}
if (!readInfo)
{
// ffmpeg读取信息
Logger.WarnMarkUp(ResString.readingInfo);
mediaInfos = await MediainfoUtil.ReadInfoAsync(DownloaderConfig.MyOptions.FFmpegBinaryPath!, result!.ActualFilePath);
mediaInfos.ForEach(info => Logger.InfoMarkUp(info.ToStringMarkUp()));
lock (lockObj)
{
if (audioStart == null) audioStart = mediaInfos.FirstOrDefault(x => x.Type == "Audio")?.StartTime;
}
ChangeSpecInfo(streamSpec, mediaInfos, ref useAACFilter);
readInfo = true;
}
}
}
// 开始下载
var options = new ParallelOptions()
{
MaxDegreeOfParallelism = DownloaderConfig.MyOptions.ThreadCount
};
await Parallel.ForEachAsync(segments, options, async (seg, _) =>
{
// 获取文件名
var filename = GetSegmentName(seg, allHasDatetime, SamePathDic[task.Id]);
var index = seg.Index;
var path = Path.Combine(tmpDir, filename + $".{streamSpec.Extension ?? "clip"}.tmp");
var result = await Downloader.DownloadSegmentAsync(seg, path, speedContainer, headers);
FileDic[seg] = result;
if (result is { Success: true })
task.Increment(1);
// 实时解密
if (seg.IsEncrypted && DownloaderConfig.MyOptions.MP4RealTimeDecryption && result is { Success: true } && !string.IsNullOrEmpty(currentKID))
{
var enc = result.ActualFilePath;
var dec = Path.Combine(Path.GetDirectoryName(enc)!, Path.GetFileNameWithoutExtension(enc) + "_dec" + Path.GetExtension(enc));
var dResult = await MP4DecryptUtil.DecryptAsync(decryptEngine, decryptionBinaryPath, DownloaderConfig.MyOptions.Keys, enc, dec, currentKID, mp4InitFile);
if (dResult)
{
File.Delete(enc);
result.ActualFilePath = dec;
}
}
});
// 自动修复VTT raw字幕
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec is { MediaType: Common.Enum.MediaType.SUBTITLES, Extension: not null } && streamSpec.Extension.Contains("vtt"))
{
// 排序字幕并修正时间戳
var keys = FileDic.Keys.OrderBy(k => k.Index).ToList();
foreach (var seg in keys)
{
var vttContent = await File.ReadAllTextAsync(FileDic[seg]!.ActualFilePath);
var waitCount = 0;
while (DownloaderConfig.MyOptions.LiveFixVttByAudio && audioStart == null && waitCount++ < 5)
{
await Task.Delay(1000);
}
var subOffset = audioStart != null ? (long)audioStart.Value.TotalMilliseconds : 0L;
var vtt = WebVttSub.Parse(vttContent, subOffset);
// 手动计算MPEGTS
if (currentVtt.MpegtsTimestamp == 0 && vtt.MpegtsTimestamp == 0)
{
vtt.MpegtsTimestamp = 90000 * (long)keys.Where(s => s.Index < seg.Index).Sum(s => s.Duration);
}
if (firstSub) { currentVtt = vtt; firstSub = false; }
else currentVtt.AddCuesFromOne(vtt);
}
}
// 自动修复VTT mp4字幕
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec.MediaType == Common.Enum.MediaType.SUBTITLES
&& streamSpec.Codecs != "stpp" && streamSpec.Extension != null && streamSpec.Extension.Contains("m4s"))
{
var initFile = FileDic.Values.FirstOrDefault(v => Path.GetFileName(v!.ActualFilePath).StartsWith("_init"));
var iniFileBytes = File.ReadAllBytes(initFile!.ActualFilePath);
var (sawVtt, timescale) = MP4VttUtil.CheckInit(iniFileBytes);
if (sawVtt)
{
var mp4s = FileDic.OrderBy(s => s.Key.Index).Select(s => s.Value).Select(v => v!.ActualFilePath).Where(p => p.EndsWith(".m4s")).ToArray();
if (firstSub)
{
currentVtt = MP4VttUtil.ExtractSub(mp4s, timescale);
firstSub = false;
}
else
{
var vtt = MP4VttUtil.ExtractSub(mp4s, timescale);
currentVtt.AddCuesFromOne(vtt);
}
}
}
// 自动修复TTML raw字幕
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec is { MediaType: Common.Enum.MediaType.SUBTITLES, Extension: not null } && streamSpec.Extension.Contains("ttml"))
{
var keys = FileDic.OrderBy(s => s.Key.Index).Where(v => v.Value!.ActualFilePath.EndsWith(".m4s")).Select(s => s.Key).ToList();
if (firstSub)
{
if (baseTimestamp != 0)
{
var total = segmentsDuration;
baseTimestamp -= (long)TimeSpan.FromSeconds(total).TotalMilliseconds;
}
var first = true;
foreach (var seg in keys)
{
var vtt = MP4TtmlUtil.ExtractFromTTML(FileDic[seg]!.ActualFilePath, 0, baseTimestamp);
// 手动计算MPEGTS
if (currentVtt.MpegtsTimestamp == 0 && vtt.MpegtsTimestamp == 0)
{
vtt.MpegtsTimestamp = 90000 * (long)keys.Where(s => s.Index < seg.Index).Sum(s => s.Duration);
}
if (first) { currentVtt = vtt; first = false; }
else currentVtt.AddCuesFromOne(vtt);
}
firstSub = false;
}
else
{
foreach (var seg in keys)
{
var vtt = MP4TtmlUtil.ExtractFromTTML(FileDic[seg]!.ActualFilePath, 0, baseTimestamp);
// 手动计算MPEGTS
if (currentVtt.MpegtsTimestamp == 0 && vtt.MpegtsTimestamp == 0)
{
vtt.MpegtsTimestamp = 90000 * (RecordedDurDic[task.Id] + (long)keys.Where(s => s.Index < seg.Index).Sum(s => s.Duration));
}
currentVtt.AddCuesFromOne(vtt);
}
}
}
// 自动修复TTML mp4字幕
if (DownloaderConfig.MyOptions.AutoSubtitleFix && streamSpec is { MediaType: Common.Enum.MediaType.SUBTITLES, Extension: not null } && streamSpec.Extension.Contains("m4s")
&& streamSpec.Codecs != null && streamSpec.Codecs.Contains("stpp"))
{
// sawTtml暂时不判断
// var initFile = FileDic.Values.Where(v => Path.GetFileName(v!.ActualFilePath).StartsWith("_init")).FirstOrDefault();
// var iniFileBytes = File.ReadAllBytes(initFile!.ActualFilePath);
// var sawTtml = MP4TtmlUtil.CheckInit(iniFileBytes);
var keys = FileDic.OrderBy(s => s.Key.Index).Where(v => v.Value!.ActualFilePath.EndsWith(".m4s")).Select(s => s.Key);
if (firstSub)
{
if (baseTimestamp != 0)
{
var total = segmentsDuration;
baseTimestamp -= (long)TimeSpan.FromSeconds(total).TotalMilliseconds;
}
var first = true;
foreach (var seg in keys)
{
var vtt = MP4TtmlUtil.ExtractFromMp4(FileDic[seg]!.ActualFilePath, 0, baseTimestamp);
// 手动计算MPEGTS
if (currentVtt.MpegtsTimestamp == 0 && vtt.MpegtsTimestamp == 0)
{
vtt.MpegtsTimestamp = 90000 * (long)keys.Where(s => s.Index < seg.Index).Sum(s => s.Duration);
}
if (first) { currentVtt = vtt; first = false; }
else currentVtt.AddCuesFromOne(vtt);
}
firstSub = false;
}
else
{
foreach (var seg in keys)
{
var vtt = MP4TtmlUtil.ExtractFromMp4(FileDic[seg]!.ActualFilePath, 0, baseTimestamp);
// 手动计算MPEGTS
if (currentVtt.MpegtsTimestamp == 0 && vtt.MpegtsTimestamp == 0)
{
vtt.MpegtsTimestamp = 90000 * (RecordedDurDic[task.Id] + (long)keys.Where(s => s.Index < seg.Index).Sum(s => s.Duration));
}
currentVtt.AddCuesFromOne(vtt);
}
}
}
RecordedDurDic[task.Id] += (int)segmentsDuration;
/*// 写出m3u8
if (DownloaderConfig.MyOptions.LiveWriteHLS)
{
var _saveDir = DownloaderConfig.MyOptions.SaveDir ?? Environment.CurrentDirectory;
var _saveName = DownloaderConfig.MyOptions.SaveName ?? DateTime.Now.ToString("yyyyMMddHHmmss");
await StreamingUtil.WriteStreamListAsync(FileDic, task.Id, 0, _saveName, _saveDir);
}*/
// 合并逻辑
if (DownloaderConfig.MyOptions.LiveRealTimeMerge)
{
// 合并
var outputExt = "." + streamSpec.Extension;
if (streamSpec.Extension == null) outputExt = ".ts";
else if (streamSpec is { MediaType: MediaType.AUDIO, Extension: "m4s" }) outputExt = ".m4a";
else if (streamSpec.MediaType != MediaType.SUBTITLES && streamSpec.Extension == "m4s") outputExt = ".mp4";
else if (streamSpec.MediaType == MediaType.SUBTITLES)
{
outputExt = DownloaderConfig.MyOptions.SubtitleFormat == Enum.SubtitleFormat.SRT ? ".srt" : ".vtt";
}
var output = Path.Combine(saveDir, saveName + outputExt);
// 移除无效片段
var badKeys = FileDic.Where(i => i.Value == null).Select(i => i.Key);
foreach (var badKey in badKeys)
{
FileDic!.Remove(badKey, out _);
}
// 设置输出流
if (fileOutputStream == null)
{
// 检测目标文件是否存在
while (File.Exists(output))
{
Logger.WarnMarkUp($"{Path.GetFileName(output)} => {Path.GetFileName(output = Path.ChangeExtension(output, $"copy" + Path.GetExtension(output)))}");
}
if (!DownloaderConfig.MyOptions.LivePipeMux || streamSpec.MediaType == MediaType.SUBTITLES)
{
fileOutputStream = new FileStream(output, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.Read);
}
else
{
// 创建管道
output = Path.ChangeExtension(output, ".ts");
var pipeName = $"RE_pipe_{Guid.NewGuid()}";
fileOutputStream = PipeUtil.CreatePipe(pipeName);
Logger.InfoMarkUp($"{ResString.namedPipeCreated} [cyan]{pipeName.EscapeMarkup()}[/]");
PipeSteamNamesDic[task.Id] = pipeName;
if (PipeSteamNamesDic.Count == SelectedSteams.Count(x => x.MediaType != MediaType.SUBTITLES))
{
var names = PipeSteamNamesDic.OrderBy(i => i.Key).Select(k => k.Value).ToArray();
Logger.WarnMarkUp($"{ResString.namedPipeMux} [deepskyblue1]{Path.GetFileName(output).EscapeMarkup()}[/]");
var t = PipeUtil.StartPipeMuxAsync(DownloaderConfig.MyOptions.FFmpegBinaryPath!, names, output);
}
// Windows only
if (OperatingSystem.IsWindows())
await (fileOutputStream as NamedPipeServerStream)!.WaitForConnectionAsync();
}
}
if (streamSpec.MediaType != MediaType.SUBTITLES)
{
var initResult = streamSpec.Playlist!.MediaInit != null ? FileDic[streamSpec.Playlist!.MediaInit!]! : null;
var files = FileDic.Where(f => f.Key != streamSpec.Playlist!.MediaInit).OrderBy(s => s.Key.Index).Select(f => f.Value).Select(v => v!.ActualFilePath).ToArray();
if (initResult != null && mp4InitFile != "")
{
// shaka/ffmpeg实时解密不需要init文件用于合并，mp4decrpyt需要
if (string.IsNullOrEmpty(currentKID) || decryptEngine == DecryptEngine.MP4DECRYPT)
{
files = [initResult.ActualFilePath, ..files];
}
}
foreach (var inputFilePath in files)
{
using (var inputStream = File.OpenRead(inputFilePath))
{
inputStream.CopyTo(fileOutputStream);
}
}
if (!DownloaderConfig.MyOptions.LiveKeepSegments)
{
foreach (var inputFilePath in files.Where(x => !Path.GetFileName(x).StartsWith("_init")))
{
File.Delete(inputFilePath);
}
}
FileDic.Clear();
if (initResult != null)
{
FileDic[streamSpec.Playlist!.MediaInit!] = initResult;
}
}
else
{
var initResult = streamSpec.Playlist!.MediaInit != null ? FileDic[streamSpec.Playlist!.MediaInit!]! : null;
var files = FileDic.OrderBy(s => s.Key.Index).Select(f => f.Value).Select(v => v!.ActualFilePath).ToArray();
foreach (var inputFilePath in files)
{
if (!DownloaderConfig.MyOptions.LiveKeepSegments && !Path.GetFileName(inputFilePath).StartsWith("_init"))
{
File.Delete(inputFilePath);
}
}
// 处理图形字幕
await SubtitleUtil.TryWriteImagePngsAsync(currentVtt, tmpDir);
var subText = currentVtt.ToVtt();
if (outputExt == ".srt")
{
subText = currentVtt.ToSrt();
}
var subBytes = Encoding.UTF8.GetBytes(subText);
fileOutputStream.Position = 0;
fileOutputStream.Write(subBytes);
FileDic.Clear();
if (initResult != null)
{
FileDic[streamSpec.Playlist!.MediaInit!] = initResult;
}
}
// 刷新buffer
if (fileOutputStream != null)
{
fileOutputStream.Flush();
}
}
if (STOP_FLAG && source.Count == 0)
break;
}
if (fileOutputStream == null) return true;
if (!DownloaderConfig.MyOptions.LivePipeMux)
{
// 记录所有文件信息
OutputFiles.Add(new OutputFile()
{
Index = task.Id,
FilePath = (fileOutputStream as FileStream)!.Name,
LangCode = streamSpec.Language,
Description = streamSpec.Name,
Mediainfos = mediaInfos,
MediaType = streamSpec.MediaType,
});
}
fileOutputStream.Close();
fileOutputStream.Dispose();
return true;
}
private async Task PlayListProduceAsync(Dictionary<StreamSpec, ProgressTask> dic)
{
while (!STOP_FLAG)
{
if (WAIT_SEC == 0) continue;
// 1. MPD 所有URL相同 单次请求即可获得所有轨道的信息
// 2. M3U8 所有URL不同 才需要多次请求
await Parallel.ForEachAsync(dic, async (dic, _) =>
{
var streamSpec = dic.Key;
var task = dic.Value;
// 达到上限时 不需要刷新了
if (RecordLimitReachedDic[task.Id])
return;
var allHasDatetime = streamSpec.Playlist!.MediaParts[0].MediaSegments.All(s => s.DateTime != null);
if (!SamePathDic.ContainsKey(task.Id))
{
var allName = streamSpec.Playlist!.MediaParts[0].MediaSegments.Select(s => OtherUtil.GetFileNameFromInput(s.Url, false));
var allSamePath = allName.Count() > 1 && allName.Distinct().Count() == 1;
SamePathDic[task.Id] = allSamePath;
}
// 过滤不需要下载的片段
FilterMediaSegments(streamSpec, task, allHasDatetime, SamePathDic[task.Id]);
var newList = streamSpec.Playlist!.MediaParts[0].MediaSegments;
if (newList.Count > 0)
{
task.MaxValue += newList.Count;
// 推送给消费者
await BlockDic[task.Id].SendAsync(newList);
// 更新最新链接
LastFileNameDic[task.Id] = GetSegmentName(newList.Last(), allHasDatetime, SamePathDic[task.Id]);
// 尝试更新时间戳
var dt = newList.Last().DateTime;
DateTimeDic[task.Id] = dt != null ? GetUnixTimestamp(dt.Value) : 0L;
// 累加已获取到的时长
RefreshedDurDic[task.Id] += (int)newList.Sum(s => s.Duration);
}
if (!STOP_FLAG && RefreshedDurDic[task.Id] >= DownloaderConfig.MyOptions.LiveRecordLimit?.TotalSeconds)
{
RecordLimitReachedDic[task.Id] = true;
}
// 检测时长限制
if (!STOP_FLAG && RecordLimitReachedDic.Values.All(x => x))
{
Logger.WarnMarkUp($"[darkorange3_1]{ResString.liveLimitReached}[/]");
STOP_FLAG = true;
CancellationTokenSource.Cancel();
}
});
try
{
// Logger.WarnMarkUp($"wait {waitSec}s");
if (!STOP_FLAG) await Task.Delay(WAIT_SEC * 1000, CancellationTokenSource.Token);
// 刷新列表
if (!STOP_FLAG) await StreamExtractor.RefreshPlayListAsync(dic.Keys.ToList());
}
catch (OperationCanceledException oce) when (oce.CancellationToken == CancellationTokenSource.Token)
{
// 不需要做事
}
catch (Exception e)
{
Logger.ErrorMarkUp(e);
STOP_FLAG = true;
// 停止所有Block
foreach (var target in BlockDic.Values)
{
target.Complete();
}
}
}
}
private void FilterMediaSegments(StreamSpec streamSpec, ProgressTask task, bool allHasDatetime, bool allSamePath)
{
if (string.IsNullOrEmpty(LastFileNameDic[task.Id]) && DateTimeDic[task.Id] == 0) return;
var index = -1;
var dateTime = DateTimeDic[task.Id];
var lastName = LastFileNameDic[task.Id];
// 优先使用dateTime判断
if (dateTime != 0 && streamSpec.Playlist!.MediaParts[0].MediaSegments.All(s => s.DateTime != null))
{
index = streamSpec.Playlist!.MediaParts[0].MediaSegments.FindIndex(s => GetUnixTimestamp(s.DateTime!.Value) == dateTime);
}
else
{
index = streamSpec.Playlist!.MediaParts[0].MediaSegments.FindIndex(s => GetSegmentName(s, allHasDatetime, allSamePath) == lastName);
}
if (index > -1)
{
// 修正Index
var list = streamSpec.Playlist!.MediaParts[0].MediaSegments.Skip(index + 1).ToList();
if (list.Count > 0)
{
var newMin = list.Min(s => s.Index);
var oldMax = MaxIndexDic[task.Id];
if (newMin < oldMax)
{
var offset = oldMax - newMin + 1;
foreach (var item in list)
{
item.Index += offset;
}
}
MaxIndexDic[task.Id] = list.Max(s => s.Index);
}
streamSpec.Playlist!.MediaParts[0].MediaSegments = list;
}
}
public async Task<bool> StartRecordAsync()
{
var takeLastCount = DownloaderConfig.MyOptions.LiveTakeCount;
ConcurrentDictionary<int, SpeedContainer> SpeedContainerDic = new(); // 速度计算
ConcurrentDictionary<StreamSpec, bool?> Results = new();
// 同步流
FilterUtil.SyncStreams(SelectedSteams, takeLastCount);
// 设置等待时间
if (WAIT_SEC == 0)
{
WAIT_SEC = (int)(SelectedSteams.Min(s => s.Playlist!.MediaParts[0].MediaSegments.Sum(s => s.Duration)) / 2);
WAIT_SEC -= 2; // 再提前两秒吧 留出冗余
if (DownloaderConfig.MyOptions.LiveWaitTime != null)
WAIT_SEC = DownloaderConfig.MyOptions.LiveWaitTime.Value;
if (WAIT_SEC <= 0) WAIT_SEC = 1;
Logger.WarnMarkUp($"set refresh interval to {WAIT_SEC} seconds");
}
// 如果没有选中音频 取消通过音频修复vtt时间轴
if (SelectedSteams.All(x => x.MediaType != MediaType.AUDIO))
{
DownloaderConfig.MyOptions.LiveFixVttByAudio = false;
}
/*// 写出master
if (DownloaderConfig.MyOptions.LiveWriteHLS)
{
var saveDir = DownloaderConfig.MyOptions.SaveDir ?? Environment.CurrentDirectory;
var saveName = DownloaderConfig.MyOptions.SaveName ?? DateTime.Now.ToString("yyyyMMddHHmmss");
await StreamingUtil.WriteMasterListAsync(SelectedSteams, saveName, saveDir);
}*/
var progress = CustomAnsiConsole.Console.Progress().AutoClear(true);
progress.AutoRefresh = DownloaderConfig.MyOptions.LogLevel != LogLevel.OFF;
// 进度条的列定义
var progressColumns = new ProgressColumn[]
{
new TaskDescriptionColumn() { Alignment = Justify.Left },
new RecordingDurationColumn(RecordedDurDic, RefreshedDurDic), // 时长显示
new RecordingStatusColumn(),
new PercentageColumn(),
new DownloadSpeedColumn(SpeedContainerDic), // 速度计算
new SpinnerColumn(),
};
if (DownloaderConfig.MyOptions.NoAnsiColor)
{
progressColumns = progressColumns.SkipLast(1).ToArray();
}
progress.Columns(progressColumns);
await progress.StartAsync(async ctx =>
{
// 创建任务
var dic = SelectedSteams.Select(item =>
{
var task = ctx.AddTask(item.ToShortShortString(), autoStart: false, maxValue: 0);
SpeedContainerDic[task.Id] = new SpeedContainer(); // 速度计算
// 限速设置
if (DownloaderConfig.MyOptions.MaxSpeed != null)
{
SpeedContainerDic[task.Id].SpeedLimit = DownloaderConfig.MyOptions.MaxSpeed.Value;
}
LastFileNameDic[task.Id] = "";
RecordLimitReachedDic[task.Id] = false;
DateTimeDic[task.Id] = 0L;
RecordedDurDic[task.Id] = 0;
RefreshedDurDic[task.Id] = 0;
MaxIndexDic[task.Id] = item.Playlist?.MediaParts[0].MediaSegments.LastOrDefault()?.Index ?? 0L; // 最大Index
BlockDic[task.Id] = new BufferBlock<List<MediaSegment>>();
return (item, task);
}).ToDictionary(item => item.item, item => item.task);
DownloaderConfig.MyOptions.ConcurrentDownload = true;
DownloaderConfig.MyOptions.MP4RealTimeDecryption = true;
DownloaderConfig.MyOptions.LiveRecordLimit ??= TimeSpan.MaxValue;
if (DownloaderConfig.MyOptions is { MP4RealTimeDecryption: true, DecryptionEngine: not DecryptEngine.SHAKA_PACKAGER, Keys.Length: > 0 })
Logger.WarnMarkUp($"[darkorange3_1]{ResString.realTimeDecMessage}[/]");
var limit = DownloaderConfig.MyOptions.LiveRecordLimit;
if (limit != TimeSpan.MaxValue)
Logger.WarnMarkUp($"[darkorange3_1]{ResString.liveLimit}{GlobalUtil.FormatTime((int)limit.Value.TotalSeconds)}[/]");
// 录制直播时，用户选了几个流就并发录几个
var options = new ParallelOptions()
{
MaxDegreeOfParallelism = SelectedSteams.Count
};
// 开始刷新
var producerTask = PlayListProduceAsync(dic);
await Task.Delay(200);
// 并发下载
await Parallel.ForEachAsync(dic, options, async (kp, _) =>
{
var task = kp.Value;
var consumerTask = RecordStreamAsync(kp.Key, task, SpeedContainerDic[task.Id], BlockDic[task.Id]);
Results[kp.Key] = await consumerTask;
});
});
var success = Results.Values.All(v => v == true);
// 删除临时文件夹
if (DownloaderConfig.MyOptions is { SkipMerge: false, DelAfterDone: true } && success)
{
foreach (var item in StreamExtractor.RawFiles)
{
var file = Path.Combine(DownloaderConfig.DirPrefix, item.Key);
if (File.Exists(file)) File.Delete(file);
}
OtherUtil.SafeDeleteDir(DownloaderConfig.DirPrefix);
}
// 混流
if (success && DownloaderConfig.MyOptions.MuxAfterDone && OutputFiles.Count > 0)
{
OutputFiles = OutputFiles.OrderBy(o => o.Index).ToList();
// 是否跳过字幕
if (DownloaderConfig.MyOptions.MuxOptions!.SkipSubtitle)
{
OutputFiles = OutputFiles.Where(o => o.MediaType != MediaType.SUBTITLES).ToList();
}
if (DownloaderConfig.MyOptions.MuxImports != null)
{
OutputFiles.AddRange(DownloaderConfig.MyOptions.MuxImports);
}
OutputFiles.ForEach(f => Logger.WarnMarkUp($"[grey]{Path.GetFileName(f.FilePath).EscapeMarkup()}[/]"));
var saveDir = DownloaderConfig.MyOptions.SaveDir ?? Environment.CurrentDirectory;
var ext = OtherUtil.GetMuxExtension(DownloaderConfig.MyOptions.MuxOptions.MuxFormat);
var dirName = Path.GetFileName(DownloaderConfig.DirPrefix);
var outName = $"{dirName}.MUX";
var outPath = Path.Combine(saveDir, outName);
Logger.WarnMarkUp($"Muxing to [grey]{outName.EscapeMarkup()}{ext}[/]");
var result = false;
if (DownloaderConfig.MyOptions.MuxOptions.UseMkvmerge) result = MergeUtil.MuxInputsByMkvmerge(DownloaderConfig.MyOptions.MkvmergeBinaryPath!, OutputFiles.ToArray(), outPath);
else result = MergeUtil.MuxInputsByFFmpeg(DownloaderConfig.MyOptions.FFmpegBinaryPath!, OutputFiles.ToArray(), outPath, DownloaderConfig.MyOptions.MuxOptions.MuxFormat, !DownloaderConfig.MyOptions.NoDateInfo);
// 完成后删除各轨道文件
if (result)
{
if (!DownloaderConfig.MyOptions.MuxOptions.KeepFiles)
{
Logger.WarnMarkUp("[grey]Cleaning files...[/]");
OutputFiles.ForEach(f => File.Delete(f.FilePath));
var tmpDir = DownloaderConfig.MyOptions.TmpDir ?? Environment.CurrentDirectory;
OtherUtil.SafeDeleteDir(tmpDir);
}
}
else
{
success = false;
Logger.ErrorMarkUp($"Mux failed");
}
// 判断是否要改名
var newPath = Path.ChangeExtension(outPath, ext);
if (result && !File.Exists(newPath))
{
Logger.WarnMarkUp($"Rename to [grey]{Path.GetFileName(newPath).EscapeMarkup()}[/]");
File.Move(outPath + ext, newPath);
}
}
return success;
}
}

9
src/N_m3u8DL-RE/Downloader/IDownloader.cs Normal file
View File

@ -0,0 +1,9 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Entity;
namespace N_m3u8DL_RE.Downloader;
internal interface IDownloader
{
Task<DownloadResult?> DownloadSegmentAsync(MediaSegment segment, string savePath, SpeedContainer speedContainer, Dictionary<string, string>? headers = null);
}

154
src/N_m3u8DL-RE/Downloader/SimpleDownloader.cs Normal file
View File

@ -0,0 +1,154 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Config;
using N_m3u8DL_RE.Crypto;
using N_m3u8DL_RE.Entity;
using N_m3u8DL_RE.Util;
using Spectre.Console;
namespace N_m3u8DL_RE.Downloader;
/// <summary>
/// 简单下载器
/// </summary>
internal class SimpleDownloader : IDownloader
{
DownloaderConfig DownloaderConfig;
public SimpleDownloader(DownloaderConfig config)
{
DownloaderConfig = config;
}
public async Task<DownloadResult?> DownloadSegmentAsync(MediaSegment segment, string savePath, SpeedContainer speedContainer, Dictionary<string, string>? headers = null)
{
var url = segment.Url;
var (des, dResult) = await DownClipAsync(url, savePath, speedContainer, segment.StartRange, segment.StopRange, headers, DownloaderConfig.MyOptions.DownloadRetryCount);
if (dResult is { Success: true } && dResult.ActualFilePath != des)
{
switch (segment.EncryptInfo.Method)
{
case EncryptMethod.AES_128:
{
var key = segment.EncryptInfo.Key;
var iv = segment.EncryptInfo.IV;
AESUtil.AES128Decrypt(dResult.ActualFilePath, key!, iv!);
break;
}
case EncryptMethod.AES_128_ECB:
{
var key = segment.EncryptInfo.Key;
var iv = segment.EncryptInfo.IV;
AESUtil.AES128Decrypt(dResult.ActualFilePath, key!, iv!, System.Security.Cryptography.CipherMode.ECB);
break;
}
case EncryptMethod.CHACHA20:
{
var key = segment.EncryptInfo.Key;
var nonce = segment.EncryptInfo.IV;
var fileBytes = File.ReadAllBytes(dResult.ActualFilePath);
var decrypted = ChaCha20Util.DecryptPer1024Bytes(fileBytes, key!, nonce!);
await File.WriteAllBytesAsync(dResult.ActualFilePath, decrypted);
break;
}
case EncryptMethod.SAMPLE_AES_CTR:
// throw new NotSupportedException("SAMPLE-AES-CTR");
break;
}
// Image头处理
if (dResult.ImageHeader)
{
await ImageHeaderUtil.ProcessAsync(dResult.ActualFilePath);
}
// Gzip解压
if (dResult.GzipHeader)
{
await OtherUtil.DeGzipFileAsync(dResult.ActualFilePath);
}
// 处理完成后改名
File.Move(dResult.ActualFilePath, des);
dResult.ActualFilePath = des;
}
return dResult;
}
private async Task<(string des, DownloadResult? dResult)> DownClipAsync(string url, string path, SpeedContainer speedContainer, long? fromPosition, long? toPosition, Dictionary<string, string>? headers = null, int retryCount = 3)
{
CancellationTokenSource? cancellationTokenSource = null;
retry:
try
{
cancellationTokenSource = new();
var des = Path.ChangeExtension(path, null);
// 已下载跳过
if (File.Exists(des))
{
speedContainer.Add(new FileInfo(des).Length);
return (des, new DownloadResult() { ActualContentLength = 0, ActualFilePath = des });
}
// 已解密跳过
var dec = Path.Combine(Path.GetDirectoryName(des)!, Path.GetFileNameWithoutExtension(des) + "_dec" + Path.GetExtension(des));
if (File.Exists(dec))
{
speedContainer.Add(new FileInfo(dec).Length);
return (dec, new DownloadResult() { ActualContentLength = 0, ActualFilePath = dec });
}
// 另起线程进行监控
var cts = cancellationTokenSource;
using var watcher = Task.Factory.StartNew(async () =>
{
while (true)
{
if (cts.IsCancellationRequested) break;
if (speedContainer.ShouldStop)
{
cts.Cancel();
Logger.DebugMarkUp("Cancel...");
break;
}
await Task.Delay(500);
}
});
// 调用下载
var result = await DownloadUtil.DownloadToFileAsync(url, path, speedContainer, cancellationTokenSource, headers, fromPosition, toPosition);
return (des, result);
throw new Exception("please retry");
}
catch (Exception ex)
{
Logger.DebugMarkUp($"[grey]{ex.Message.EscapeMarkup()} retryCount: {retryCount}[/]");
Logger.Debug(url + " " + ex);
Logger.Extra($"Ah oh!{Environment.NewLine}RetryCount => {retryCount}{Environment.NewLine}Exception => {ex.Message}{Environment.NewLine}Url => {url}");
if (retryCount-- > 0)
{
await Task.Delay(1000);
goto retry;
}
else
{
Logger.Extra($"The retry attempts have been exhausted and the download of this segment has failed.{Environment.NewLine}Exception => {ex.Message}{Environment.NewLine}Url => {url}");
Logger.WarnMarkUp($"[grey]{ex.Message.EscapeMarkup()}[/]");
}
// throw new Exception("download failed", ex);
return default;
}
finally
{
if (cancellationTokenSource != null)
{
// 调用后销毁
cancellationTokenSource.Dispose();
cancellationTokenSource = null;
}
}
}
}

16
src/N_m3u8DL-RE/Entity/CustomRange.cs Normal file
View File

@ -0,0 +1,16 @@
namespace N_m3u8DL_RE.Entity;
public class CustomRange
{
public required string InputStr { get; set; }
public double? StartSec { get; set; }
public double? EndSec { get; set; }
public long? StartSegIndex { get; set; }
public long? EndSegIndex { get; set;}
public override string? ToString()
{
return $"StartSec: {StartSec}, EndSec: {EndSec}, StartSegIndex: {StartSegIndex}, EndSegIndex: {EndSegIndex}";
}
}

11
src/N_m3u8DL-RE/Entity/DownloadResult.cs Normal file
View File

@ -0,0 +1,11 @@
namespace N_m3u8DL_RE.Entity;
internal class DownloadResult
{
public bool Success => (ActualContentLength != null && RespContentLength != null) ? (RespContentLength == ActualContentLength) : (ActualContentLength != null);
public long? RespContentLength { get; set; }
public long? ActualContentLength { get; set; }
public bool ImageHeader { get; set; } = false; // 图片伪装
public bool GzipHeader { get; set; } = false; // GZip压缩
public required string ActualFilePath { get; set; }
}

27
src/N_m3u8DL-RE/Entity/Mediainfo.cs Normal file
View File

@ -0,0 +1,27 @@
using Spectre.Console;
namespace N_m3u8DL_RE.Entity;
internal class Mediainfo
{
public string? Id { get; set; }
public string? Text { get; set; }
public string? BaseInfo { get; set; }
public string? Bitrate { get; set; }
public string? Resolution { get; set; }
public string? Fps { get; set; }
public string? Type { get; set; }
public TimeSpan? StartTime { get; set; }
public bool DolbyVison { get; set; }
public bool HDR { get; set; }
public override string? ToString()
{
return $"{(string.IsNullOrEmpty(Id) ? "NaN" : Id)}: " + string.Join(", ", new List<string?> { Type, BaseInfo, Resolution, Fps, Bitrate }.Where(i => !string.IsNullOrEmpty(i)));
}
public string ToStringMarkUp()
{
return "[steelblue]" + ToString().EscapeMarkup() + ((HDR && !DolbyVison) ? " [darkorange3_1][[HDR]][/]" : "") + (DolbyVison ? " [darkorange3_1][[DOVI]][/]" : "") + "[/]";
}
}

12
src/N_m3u8DL-RE/Entity/MuxOptions.cs Normal file
View File

@ -0,0 +1,12 @@
using N_m3u8DL_RE.Enum;
namespace N_m3u8DL_RE.Entity;
internal class MuxOptions
{
public bool UseMkvmerge { get; set; } = false;
public MuxFormat MuxFormat { get; set; } = MuxFormat.MP4;
public bool KeepFiles { get; set; } = false;
public bool SkipSubtitle { get; set; } = false;
public string? BinPath { get; set; }
}

13
src/N_m3u8DL-RE/Entity/OutputFile.cs Normal file
View File

@ -0,0 +1,13 @@
using N_m3u8DL_RE.Common.Enum;
namespace N_m3u8DL_RE.Entity;
internal class OutputFile
{
public MediaType? MediaType { get; set; }
public required int Index { get; set; }
public required string FilePath { get; set; }
public string? LangCode { get; set; }
public string? Description { get; set; }
public List<Mediainfo> Mediainfos { get; set; } = [];
}

49
src/N_m3u8DL-RE/Entity/SpeedContainer.cs Normal file
View File

@ -0,0 +1,49 @@
namespace N_m3u8DL_RE.Entity;
internal class SpeedContainer
{
public bool SingleSegment { get; set; } = false;
public long NowSpeed { get; set; } = 0L; // 当前每秒速度
public long SpeedLimit { get; set; } = long.MaxValue; // 限速设置
public long? ResponseLength { get; set; }
public long RDownloaded => _Rdownloaded;
private int _zeroSpeedCount = 0;
public int LowSpeedCount => _zeroSpeedCount;
public bool ShouldStop => LowSpeedCount >= 20;
///////////////////////////////////////////////////
private long _downloaded = 0;
private long _Rdownloaded = 0;
public long Downloaded => _downloaded;
public int AddLowSpeedCount()
{
return Interlocked.Add(ref _zeroSpeedCount, 1);
}
public int ResetLowSpeedCount()
{
return Interlocked.Exchange(ref _zeroSpeedCount, 0);
}
public long Add(long size)
{
Interlocked.Add(ref _Rdownloaded, size);
return Interlocked.Add(ref _downloaded, size);
}
public void Reset()
{
Interlocked.Exchange(ref _downloaded, 0);
}
public void ResetVars()
{
Reset();
ResetLowSpeedCount();
SingleSegment = false;
ResponseLength = null;
_Rdownloaded = 0L;
}
}

51
src/N_m3u8DL-RE/Entity/StreamFilter.cs Normal file
View File

@ -0,0 +1,51 @@
using N_m3u8DL_RE.Common.Enum;
using System.Text;
using System.Text.RegularExpressions;
namespace N_m3u8DL_RE.Entity;
public class StreamFilter
{
public Regex? GroupIdReg { get; set; }
public Regex? LanguageReg { get; set; }
public Regex? NameReg { get; set; }
public Regex? CodecsReg { get; set; }
public Regex? ResolutionReg { get; set; }
public Regex? FrameRateReg { get; set; }
public Regex? ChannelsReg { get; set; }
public Regex? VideoRangeReg { get; set; }
public Regex? UrlReg { get; set; }
public long? SegmentsMinCount { get; set; }
public long? SegmentsMaxCount { get; set; }
public double? PlaylistMinDur { get; set; }
public double? PlaylistMaxDur { get; set; }
public int? BandwidthMin { get; set; }
public int? BandwidthMax { get; set; }
public RoleType? Role { get; set; }
public string For { get; set; } = "best";
public override string? ToString()
{
var sb = new StringBuilder();
if (GroupIdReg != null) sb.Append($"GroupIdReg: {GroupIdReg} ");
if (LanguageReg != null) sb.Append($"LanguageReg: {LanguageReg} ");
if (NameReg != null) sb.Append($"NameReg: {NameReg} ");
if (CodecsReg != null) sb.Append($"CodecsReg: {CodecsReg} ");
if (ResolutionReg != null) sb.Append($"ResolutionReg: {ResolutionReg} ");
if (FrameRateReg != null) sb.Append($"FrameRateReg: {FrameRateReg} ");
if (ChannelsReg != null) sb.Append($"ChannelsReg: {ChannelsReg} ");
if (VideoRangeReg != null) sb.Append($"VideoRangeReg: {VideoRangeReg} ");
if (UrlReg != null) sb.Append($"UrlReg: {UrlReg} ");
if (SegmentsMinCount != null) sb.Append($"SegmentsMinCount: {SegmentsMinCount} ");
if (SegmentsMaxCount != null) sb.Append($"SegmentsMaxCount: {SegmentsMaxCount} ");
if (PlaylistMinDur != null) sb.Append($"PlaylistMinDur: {PlaylistMinDur} ");
if (PlaylistMaxDur != null) sb.Append($"PlaylistMaxDur: {PlaylistMaxDur} ");
if (BandwidthMin != null) sb.Append($"{nameof(BandwidthMin)}: {BandwidthMin} ");
if (BandwidthMax != null) sb.Append($"{nameof(BandwidthMax)}: {BandwidthMax} ");
if (Role.HasValue) sb.Append($"Role: {Role} ");
return sb + $"For: {For}";
}
}

8
src/N_m3u8DL-RE/Enum/DecryptEngine.cs Normal file
View File

@ -0,0 +1,8 @@
namespace N_m3u8DL_RE.Enum;
internal enum DecryptEngine
{
MP4DECRYPT,
SHAKA_PACKAGER,
FFMPEG,
}

8
src/N_m3u8DL-RE/Enum/MuxFormat.cs Normal file
View File

@ -0,0 +1,8 @@
namespace N_m3u8DL_RE.Enum;
internal enum MuxFormat
{
MP4,
MKV,
TS,
}

7
src/N_m3u8DL-RE/Enum/SubtitleFormat.cs Normal file
View File

@ -0,0 +1,7 @@
namespace N_m3u8DL_RE.Enum;
internal enum SubtitleFormat
{
VTT,
SRT
}

23
src/N_m3u8DL-RE/N_m3u8DL-RE.csproj Normal file
View File

@ -0,0 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0</TargetFramework>
<RootNamespace>N_m3u8DL_RE</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<LangVersion>preview</LangVersion>
<Nullable>enable</Nullable>
<Version>0.3.0</Version>
<Platforms>AnyCPU;x64</Platforms>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\N_m3u8DL-RE.Parser\N_m3u8DL-RE.Parser.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="NiL.JS" Version="2.5.1684" />
<PackageReference Include="System.CommandLine" Version="2.0.0-beta4.22272.1" />
</ItemGroup>
</Project>

21
src/N_m3u8DL-RE/Processor/DemoProcessor.cs Normal file
View File

@ -0,0 +1,21 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Parser.Processor;
namespace N_m3u8DL_RE.Processor;
internal class DemoProcessor : ContentProcessor
{
public override bool CanProcess(ExtractorType extractorType, string rawText, ParserConfig parserConfig)
{
return extractorType == ExtractorType.MPEG_DASH && parserConfig.Url.Contains("bitmovin");
}
public override string Process(string rawText, ParserConfig parserConfig)
{
Logger.InfoMarkUp("[red]Match bitmovin![/]");
return rawText;
}
}

25
src/N_m3u8DL-RE/Processor/DemoProcessor2.cs Normal file
View File

@ -0,0 +1,25 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Parser.Processor;
using N_m3u8DL_RE.Parser.Processor.HLS;
namespace N_m3u8DL_RE.Processor;
internal class DemoProcessor2 : KeyProcessor
{
public override bool CanProcess(ExtractorType extractorType, string keyLine, string m3u8Url, string m3u8Content, ParserConfig parserConfig)
{
return extractorType == ExtractorType.HLS && parserConfig.Url.Contains("playertest.longtailvideo.com");
}
public override EncryptInfo Process(string keyLine, string m3u8Url, string m3u8Content, ParserConfig parserConfig)
{
Logger.InfoMarkUp($"[white on green]My Key Processor => {keyLine}[/]");
var info = new DefaultHLSKeyProcessor().Process(keyLine, m3u8Url, m3u8Content, parserConfig);
Logger.InfoMarkUp("[red]" + HexUtil.BytesToHex(info.Key!, " ") + "[/]");
return info;
}
}

216
src/N_m3u8DL-RE/Processor/NowehoryzontyUrlProcessor.cs Normal file
View File

@ -0,0 +1,216 @@
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Parser.Processor;
using N_m3u8DL_RE.Parser.Util;
using NiL.JS.BaseLibrary;
using NiL.JS.Core;
using NiL.JS.Extensions;
namespace N_m3u8DL_RE.Processor;
// "https://1429754964.rsc.cdn77.org/r/nh22/2022/VNUS_DE_NYKE/19_07_22_2302_skt/h264.mpd?secure=mSvVfvuciJt9wufUyzuBnA==,1658505709774" --urlprocessor-args "nowehoryzonty:timeDifference=-2274,filminfo.secureToken=vx54axqjal4f0yy2"
internal class NowehoryzontyUrlProcessor : UrlProcessor
{
private static string START = "nowehoryzonty:";
private static string? TimeDifferenceStr = null;
private static int? TimeDifference = null;
private static string? SecureToken = null;
private static bool LOG = false;
private static Function? Function = null;
public override bool CanProcess(ExtractorType extractorType, string oriUrl, ParserConfig parserConfig)
{
if (extractorType == ExtractorType.MPEG_DASH && parserConfig.UrlProcessorArgs != null && parserConfig.UrlProcessorArgs.StartsWith(START))
{
if (!LOG)
{
Logger.WarnMarkUp($"[white on green]www.nowehoryzonty.pl[/] matched! waiting for calc...");
LOG = true;
}
var context = new Context();
context.Eval(JS);
Function = context.GetVariable("md5").As<Function>();
var argLine = parserConfig.UrlProcessorArgs![START.Length..];
TimeDifferenceStr = ParserUtil.GetAttribute(argLine, "timeDifference");
SecureToken = ParserUtil.GetAttribute(argLine, "filminfo.secureToken");
if (TimeDifferenceStr != null && SecureToken != null)
{
TimeDifference = Convert.ToInt32(TimeDifferenceStr);
}
return true;
}
return false;
}
public override string Process(string oriUrl, ParserConfig parserConfig)
{
var a = new Uri(oriUrl).AbsolutePath;
var n = oriUrl + "?secure=" + Calc(a);
return n;
}
private static string Calc(string a)
{
string returnStr = Function!.Call(new Arguments { a, SecureToken, TimeDifference }).ToString();
return returnStr;
}
////https://www.nowehoryzonty.pl/packed/videonho.js?v=1114377281:formatted
private static readonly string JS = """
var p = function(f, e) {
var d = f[0]
, a = f[1]
, b = f[2]
, c = f[3];
d = h(d, a, b, c, e[0], 7, -680876936);
c = h(c, d, a, b, e[1], 12, -389564586);
b = h(b, c, d, a, e[2], 17, 606105819);
a = h(a, b, c, d, e[3], 22, -1044525330);
d = h(d, a, b, c, e[4], 7, -176418897);
c = h(c, d, a, b, e[5], 12, 1200080426);
b = h(b, c, d, a, e[6], 17, -1473231341);
a = h(a, b, c, d, e[7], 22, -45705983);
d = h(d, a, b, c, e[8], 7, 1770035416);
c = h(c, d, a, b, e[9], 12, -1958414417);
b = h(b, c, d, a, e[10], 17, -42063);
a = h(a, b, c, d, e[11], 22, -1990404162);
d = h(d, a, b, c, e[12], 7, 1804603682);
c = h(c, d, a, b, e[13], 12, -40341101);
b = h(b, c, d, a, e[14], 17, -1502002290);
a = h(a, b, c, d, e[15], 22, 1236535329);
d = k(d, a, b, c, e[1], 5, -165796510);
c = k(c, d, a, b, e[6], 9, -1069501632);
b = k(b, c, d, a, e[11], 14, 643717713);
a = k(a, b, c, d, e[0], 20, -373897302);
d = k(d, a, b, c, e[5], 5, -701558691);
c = k(c, d, a, b, e[10], 9, 38016083);
b = k(b, c, d, a, e[15], 14, -660478335);
a = k(a, b, c, d, e[4], 20, -405537848);
d = k(d, a, b, c, e[9], 5, 568446438);
c = k(c, d, a, b, e[14], 9, -1019803690);
b = k(b, c, d, a, e[3], 14, -187363961);
a = k(a, b, c, d, e[8], 20, 1163531501);
d = k(d, a, b, c, e[13], 5, -1444681467);
c = k(c, d, a, b, e[2], 9, -51403784);
b = k(b, c, d, a, e[7], 14, 1735328473);
a = k(a, b, c, d, e[12], 20, -1926607734);
d = g(a ^ b ^ c, d, a, e[5], 4, -378558);
c = g(d ^ a ^ b, c, d, e[8], 11, -2022574463);
b = g(c ^ d ^ a, b, c, e[11], 16, 1839030562);
a = g(b ^ c ^ d, a, b, e[14], 23, -35309556);
d = g(a ^ b ^ c, d, a, e[1], 4, -1530992060);
c = g(d ^ a ^ b, c, d, e[4], 11, 1272893353);
b = g(c ^ d ^ a, b, c, e[7], 16, -155497632);
a = g(b ^ c ^ d, a, b, e[10], 23, -1094730640);
d = g(a ^ b ^ c, d, a, e[13], 4, 681279174);
c = g(d ^ a ^ b, c, d, e[0], 11, -358537222);
b = g(c ^ d ^ a, b, c, e[3], 16, -722521979);
a = g(b ^ c ^ d, a, b, e[6], 23, 76029189);
d = g(a ^ b ^ c, d, a, e[9], 4, -640364487);
c = g(d ^ a ^ b, c, d, e[12], 11, -421815835);
b = g(c ^ d ^ a, b, c, e[15], 16, 530742520);
a = g(b ^ c ^ d, a, b, e[2], 23, -995338651);
d = l(d, a, b, c, e[0], 6, -198630844);
c = l(c, d, a, b, e[7], 10, 1126891415);
b = l(b, c, d, a, e[14], 15, -1416354905);
a = l(a, b, c, d, e[5], 21, -57434055);
d = l(d, a, b, c, e[12], 6, 1700485571);
c = l(c, d, a, b, e[3], 10, -1894986606);
b = l(b, c, d, a, e[10], 15, -1051523);
a = l(a, b, c, d, e[1], 21, -2054922799);
d = l(d, a, b, c, e[8], 6, 1873313359);
c = l(c, d, a, b, e[15], 10, -30611744);
b = l(b, c, d, a, e[6], 15, -1560198380);
a = l(a, b, c, d, e[13], 21, 1309151649);
d = l(d, a, b, c, e[4], 6, -145523070);
c = l(c, d, a, b, e[11], 10, -1120210379);
b = l(b, c, d, a, e[2], 15, 718787259);
a = l(a, b, c, d, e[9], 21, -343485551);
f[0] = m(d, f[0]);
f[1] = m(a, f[1]);
f[2] = m(b, f[2]);
f[3] = m(c, f[3])
}, g = function(f, e, d, a, b, c) {
e = m(m(e, f), m(a, c));
return m(e << b | e >>> 32 - b, d)
}
, h = function(f, e, d, a, b, c, n) {
return g(e & d | ~e & a, f, e, b, c, n)
}
, k = function(f, e, d, a, b, c, n) {
return g(e & a | d & ~a, f, e, b, c, n)
}
, l = function(f, e, d, a, b, c, n) {
return g(d ^ (e | ~a), f, e, b, c, n)
}, r = "0123456789abcdef".split("");
var m = function(f, e) {
return f + e & 4294967295
};
var q = function(f) {
var e = f.length, d = [1732584193, -271733879, -1732584194, 271733878], a;
for (a = 64; a <= f.length; a += 64) {
var b, c = f.substring(a - 64, a), g = [];
for (b = 0; 64 > b; b += 4)
g[b >> 2] = c.charCodeAt(b) + (c.charCodeAt(b + 1) << 8) + (c.charCodeAt(b + 2) << 16) + (c.charCodeAt(b + 3) << 24);
p(d, g)
}
f = f.substring(a - 64);
b = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0];
for (a = 0; a < f.length; a++)
b[a >> 2] |= f.charCodeAt(a) << (a % 4 << 3);
b[a >> 2] |= 128 << (a % 4 << 3);
if (55 < a)
for (p(d, b),
a = 0; 16 > a; a++)
b[a] = 0;
b[14] = 8 * e;
p(d, b);
return d
};
var md5 = function(f, e, timeDifference) {
var d = Date.now() + 6E4 + timeDifference;
e = q(d + f + e);
f = [];
for (var a = 0; a < e.length; a++) {
var b = e[a];
var c = []
, g = 4;
do
c[--g] = b & 255,
b >>= 8;
while (g);
b = c;
for (c = b.length - 1; 0 <= c; c--)
f.push(b[c])
}
g = void 0;
c = "";
for (e = a = b = 0; e < 4 * f.length / 3; g = b >> 2 * (++e & 3) & 63,
c += String.fromCharCode(g + 71 - (26 > g ? 6 : 52 > g ? 0 : 62 > g ? 75 : g ^ 63 ? 90 : 87)) + (75 == (e - 1) % 76 ? "\r\n" : ""))
e & 3 ^ 3 && (b = b << 8 ^ f[a++]);
for (; e++ & 3; )
c += "\x3d";
return c.replace(/\+/g, "-").replace(/\//g, "_") + "," + d
};
"5d41402abc4b2a76b9719d911017c592" != function(f) {
for (var e = 0; e < f.length; e++) {
for (var d = e, a = f[e], b = "", c = 0; 4 > c; c++)
b += r[a >> 8 * c + 4 & 15] + r[a >> 8 * c & 15];
f[d] = b
}
return f.join("")
}(q("hello")) && (m = function(f, e) {
var d = (f & 65535) + (e & 65535);
return (f >> 16) + (e >> 16) + (d >> 16) << 16 | d & 65535
}
)
//console.log(md5('/r/nh22/2022/VNUS_DE_NYKE/19_07_22_2302_skt/h264.mpd','vx54axqjal4f0yy2',-2274));
//console.log(md5('/r/nh22/2022/VNUS_DE_NYKE/19_07_22_2302_skt/subtitle_pl/34.m4s','vx54axqjal4f0yy2',-2274));
""";
}

482
src/N_m3u8DL-RE/Program.cs Normal file
View File

@ -0,0 +1,482 @@
using System.Globalization;
using N_m3u8DL_RE.Parser.Config;
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Parser;
using Spectre.Console;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Common.Log;
using System.Text;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Processor;
using N_m3u8DL_RE.Config;
using N_m3u8DL_RE.Util;
using N_m3u8DL_RE.DownloadManager;
using N_m3u8DL_RE.CommandLine;
using System.Net;
using N_m3u8DL_RE.Enum;
namespace N_m3u8DL_RE;
internal class Program
{
static async Task Main(string[] args)
{
// 处理NT6.0及以下System.CommandLine报错CultureNotFound问题
if (OperatingSystem.IsWindows())
{
var osVersion = Environment.OSVersion.Version;
if (osVersion.Major < 6 || osVersion is { Major: 6, Minor: 0 })
{
Environment.SetEnvironmentVariable("DOTNET_SYSTEM_GLOBALIZATION_INVARIANT", "1");
}
}
Console.CancelKeyPress += Console_CancelKeyPress;
ServicePointManager.DefaultConnectionLimit = 1024;
try { Console.CursorVisible = true; } catch { }
string loc = ResString.CurrentLoc;
string currLoc = Thread.CurrentThread.CurrentUICulture.Name;
if (currLoc is "zh-CN" or "zh-SG") loc = "zh-CN";
else if (currLoc.StartsWith("zh-")) loc = "zh-TW";
// 处理用户-h等请求
var index = -1;
var list = new List<string>(args);
if ((index = list.IndexOf("--ui-language")) != -1 && list.Count > index + 1 && new List<string> { "en-US", "zh-CN", "zh-TW" }.Contains(list[index + 1]))
{
loc = list[index + 1];
}
ResString.CurrentLoc = loc;
try
{
CultureInfo.DefaultThreadCurrentCulture = new CultureInfo(loc);
Thread.CurrentThread.CurrentUICulture = CultureInfo.GetCultureInfo(loc);
Thread.CurrentThread.CurrentCulture = CultureInfo.GetCultureInfo(loc);
}
catch
{
// Culture not work on NT6.0, so catch the exception
}
await CommandInvoker.InvokeArgs(args, DoWorkAsync);
}
private static void Console_CancelKeyPress(object? sender, ConsoleCancelEventArgs e)
{
Logger.WarnMarkUp("Force Exit...");
try
{
Console.CursorVisible = true;
if (!OperatingSystem.IsWindows())
System.Diagnostics.Process.Start("tput", "cnorm");
} catch { }
Environment.Exit(0);
}
static int GetOrder(StreamSpec streamSpec)
{
if (streamSpec.Channels == null) return 0;
var str = streamSpec.Channels.Split('/')[0];
return int.TryParse(str, out var order) ? order : 0;
}
static async Task DoWorkAsync(MyOption option)
{
HTTPUtil.AppHttpClient.Timeout = TimeSpan.FromSeconds(option.HttpRequestTimeout);
if (Console.IsOutputRedirected || Console.IsErrorRedirected)
{
option.ForceAnsiConsole = true;
option.NoAnsiColor = true;
Logger.Info(ResString.consoleRedirected);
}
CustomAnsiConsole.InitConsole(option.ForceAnsiConsole, option.NoAnsiColor);
// 检测更新
if (!option.DisableUpdateCheck)
_ = CheckUpdateAsync();
Logger.IsWriteFile = !option.NoLog;
Logger.InitLogFile();
Logger.LogLevel = option.LogLevel;
Logger.Info(CommandInvoker.VERSION_INFO);
if (option.UseSystemProxy == false)
{
HTTPUtil.HttpClientHandler.UseProxy = false;
}
if (option.CustomProxy != null)
{
HTTPUtil.HttpClientHandler.Proxy = option.CustomProxy;
HTTPUtil.HttpClientHandler.UseProxy = true;
}
// 检查互斥的选项
if (option is { MuxAfterDone: false, MuxImports.Count: > 0 })
{
throw new ArgumentException("MuxAfterDone disabled, MuxImports not allowed!");
}
if (option.UseShakaPackager)
{
option.DecryptionEngine = DecryptEngine.SHAKA_PACKAGER;
}
// LivePipeMux开启时 LiveRealTimeMerge必须开启
if (option is { LivePipeMux: true, LiveRealTimeMerge: false })
{
Logger.WarnMarkUp("LivePipeMux detected, forced enable LiveRealTimeMerge");
option.LiveRealTimeMerge = true;
}
// 预先检查ffmpeg
option.FFmpegBinaryPath ??= GlobalUtil.FindExecutable("ffmpeg");
if (string.IsNullOrEmpty(option.FFmpegBinaryPath) || !File.Exists(option.FFmpegBinaryPath))
{
throw new FileNotFoundException(ResString.ffmpegNotFound);
}
Logger.Extra($"ffmpeg => {option.FFmpegBinaryPath}");
// 预先检查mkvmerge
if (option is { MuxOptions.UseMkvmerge: true, MuxAfterDone: true })
{
option.MkvmergeBinaryPath ??= GlobalUtil.FindExecutable("mkvmerge");
if (string.IsNullOrEmpty(option.MkvmergeBinaryPath) || !File.Exists(option.MkvmergeBinaryPath))
{
throw new FileNotFoundException(ResString.mkvmergeNotFound);
}
Logger.Extra($"mkvmerge => {option.MkvmergeBinaryPath}");
}
// 预先检查
if (option.Keys is { Length: > 0 } || option.KeyTextFile != null)
{
if (!string.IsNullOrEmpty(option.DecryptionBinaryPath) && !File.Exists(option.DecryptionBinaryPath))
{
throw new FileNotFoundException(option.DecryptionBinaryPath);
}
switch (option.DecryptionEngine)
{
case DecryptEngine.SHAKA_PACKAGER:
{
var file = GlobalUtil.FindExecutable("shaka-packager");
var file2 = GlobalUtil.FindExecutable("packager-linux-x64");
var file3 = GlobalUtil.FindExecutable("packager-osx-x64");
var file4 = GlobalUtil.FindExecutable("packager-win-x64");
if (file == null && file2 == null && file3 == null && file4 == null)
throw new FileNotFoundException(ResString.shakaPackagerNotFound);
option.DecryptionBinaryPath = file ?? file2 ?? file3 ?? file4;
Logger.Extra($"shaka-packager => {option.DecryptionBinaryPath}");
break;
}
case DecryptEngine.MP4DECRYPT:
{
var file = GlobalUtil.FindExecutable("mp4decrypt");
if (file == null) throw new FileNotFoundException(ResString.mp4decryptNotFound);
option.DecryptionBinaryPath = file;
Logger.Extra($"mp4decrypt => {option.DecryptionBinaryPath}");
break;
}
case DecryptEngine.FFMPEG:
default:
option.DecryptionBinaryPath = option.FFmpegBinaryPath;
break;
}
}
// 默认的headers
var headers = new Dictionary<string, string>()
{
["user-agent"] = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
};
// 添加或替换用户输入的headers
foreach (var item in option.Headers)
{
headers[item.Key] = item.Value;
Logger.Extra($"User-Defined Header => {item.Key}: {item.Value}");
}
var parserConfig = new ParserConfig()
{
AppendUrlParams = option.AppendUrlParams,
UrlProcessorArgs = option.UrlProcessorArgs,
BaseUrl = option.BaseUrl!,
Headers = headers,
CustomMethod = option.CustomHLSMethod,
CustomeKey = option.CustomHLSKey,
CustomeIV = option.CustomHLSIv,
};
if (option.AllowHlsMultiExtMap)
{
parserConfig.CustomParserArgs.Add("AllowHlsMultiExtMap", "true");
}
// demo1
parserConfig.ContentProcessors.Insert(0, new DemoProcessor());
// demo2
parserConfig.KeyProcessors.Insert(0, new DemoProcessor2());
// for www.nowehoryzonty.pl
parserConfig.UrlProcessors.Insert(0, new NowehoryzontyUrlProcessor());
// 等待任务开始时间
if (option.TaskStartAt != null && option.TaskStartAt > DateTime.Now)
{
Logger.InfoMarkUp(ResString.taskStartAt + option.TaskStartAt);
while (option.TaskStartAt > DateTime.Now)
{
await Task.Delay(1000);
}
}
var url = option.Input;
// 流提取器配置
var extractor = new StreamExtractor(parserConfig);
// 从链接加载内容
await RetryUtil.WebRequestRetryAsync(async () =>
{
await extractor.LoadSourceFromUrlAsync(url);
return true;
});
// 解析流信息
var streams = await extractor.ExtractStreamsAsync();
// 全部媒体
var lists = streams.OrderBy(p => p.MediaType).ThenByDescending(p => p.Bandwidth).ThenByDescending(GetOrder).ToList();
// 基本流
var basicStreams = lists.Where(x => x.MediaType is null or MediaType.VIDEO).ToList();
// 可选音频轨道
var audios = lists.Where(x => x.MediaType == MediaType.AUDIO).ToList();
// 可选字幕轨道
var subs = lists.Where(x => x.MediaType == MediaType.SUBTITLES).ToList();
// 尝试从URL或文件读取文件名
if (string.IsNullOrEmpty(option.SaveName))
{
option.SaveName = OtherUtil.GetFileNameFromInput(option.Input);
}
// 生成文件夹
var tmpDir = Path.Combine(option.TmpDir ?? Environment.CurrentDirectory, $"{option.SaveName ?? DateTime.Now.ToString("yyyy-MM-dd_HH-mm-ss")}");
// 记录文件
extractor.RawFiles["meta.json"] = GlobalUtil.ConvertToJson(lists);
// 写出文件
await WriteRawFilesAsync(option, extractor, tmpDir);
Logger.Info(ResString.streamsInfo, lists.Count, basicStreams.Count, audios.Count, subs.Count);
foreach (var item in lists)
{
Logger.InfoMarkUp(item.ToString());
}
var selectedStreams = new List<StreamSpec>();
if (option.DropVideoFilter != null || option.DropAudioFilter != null || option.DropSubtitleFilter != null)
{
basicStreams = FilterUtil.DoFilterDrop(basicStreams, option.DropVideoFilter);
audios = FilterUtil.DoFilterDrop(audios, option.DropAudioFilter);
subs = FilterUtil.DoFilterDrop(subs, option.DropSubtitleFilter);
lists = basicStreams.Concat(audios).Concat(subs).ToList();
}
if (option.DropVideoFilter != null) Logger.Extra($"DropVideoFilter => {option.DropVideoFilter}");
if (option.DropAudioFilter != null) Logger.Extra($"DropAudioFilter => {option.DropAudioFilter}");
if (option.DropSubtitleFilter != null) Logger.Extra($"DropSubtitleFilter => {option.DropSubtitleFilter}");
if (option.VideoFilter != null) Logger.Extra($"VideoFilter => {option.VideoFilter}");
if (option.AudioFilter != null) Logger.Extra($"AudioFilter => {option.AudioFilter}");
if (option.SubtitleFilter != null) Logger.Extra($"SubtitleFilter => {option.SubtitleFilter}");
if (option.AutoSelect)
{
if (basicStreams.Count != 0)
selectedStreams.Add(basicStreams.First());
var langs = audios.DistinctBy(a => a.Language).Select(a => a.Language);
foreach (var lang in langs)
{
selectedStreams.Add(audios.Where(a => a.Language == lang).OrderByDescending(a => a.Bandwidth).ThenByDescending(GetOrder).First());
}
selectedStreams.AddRange(subs);
}
else if (option.SubOnly)
{
selectedStreams.AddRange(subs);
}
else if (option.VideoFilter != null || option.AudioFilter != null || option.SubtitleFilter != null)
{
basicStreams = FilterUtil.DoFilterKeep(basicStreams, option.VideoFilter);
audios = FilterUtil.DoFilterKeep(audios, option.AudioFilter);
subs = FilterUtil.DoFilterKeep(subs, option.SubtitleFilter);
selectedStreams = basicStreams.Concat(audios).Concat(subs).ToList();
}
else
{
// 展示交互式选择框
selectedStreams = FilterUtil.SelectStreams(lists);
}
if (selectedStreams.Count == 0)
throw new Exception(ResString.noStreamsToDownload);
// HLS: 选中流中若有没加载出playlist的，加载playlist
// DASH/MSS: 加载playlist (调用url预处理器)
if (selectedStreams.Any(s => s.Playlist == null) || extractor.ExtractorType == ExtractorType.MPEG_DASH || extractor.ExtractorType == ExtractorType.MSS)
await extractor.FetchPlayListAsync(selectedStreams);
// 直播检测
var livingFlag = selectedStreams.Any(s => s.Playlist?.IsLive == true) && !option.LivePerformAsVod;
if (livingFlag)
{
Logger.WarnMarkUp($"[white on darkorange3_1]{ResString.liveFound}[/]");
}
// 无法识别的加密方式，自动开启二进制合并
if (selectedStreams.Any(s => s.Playlist!.MediaParts.Any(p => p.MediaSegments.Any(m => m.EncryptInfo.Method == EncryptMethod.UNKNOWN))))
{
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge3}[/]");
option.BinaryMerge = true;
}
// 应用用户自定义的分片范围
if (!livingFlag)
FilterUtil.ApplyCustomRange(selectedStreams, option.CustomRange);
// 应用用户自定义的广告分片关键字
FilterUtil.CleanAd(selectedStreams, option.AdKeywords);
// 记录文件
extractor.RawFiles["meta_selected.json"] = GlobalUtil.ConvertToJson(selectedStreams);
Logger.Info(ResString.selectedStream);
foreach (var item in selectedStreams)
{
Logger.InfoMarkUp(item.ToString());
}
// 写出文件
await WriteRawFilesAsync(option, extractor, tmpDir);
if (option.SkipDownload)
{
return;
}
#if DEBUG
Console.WriteLine("Press any key to continue...");
Console.ReadKey();
#endif
Logger.InfoMarkUp(ResString.saveName + $"[deepskyblue1]{option.SaveName.EscapeMarkup()}[/]");
// 开始MuxAfterDone后自动使用二进制版
if (option is { BinaryMerge: false, MuxAfterDone: true })
{
option.BinaryMerge = true;
Logger.WarnMarkUp($"[darkorange3_1]{ResString.autoBinaryMerge6}[/]");
}
// 下载配置
var downloadConfig = new DownloaderConfig()
{
MyOptions = option,
DirPrefix = tmpDir,
Headers = parserConfig.Headers, // 使用命令行解析得到的Headers
};
var result = false;
if (extractor.ExtractorType == ExtractorType.HTTP_LIVE)
{
var sldm = new HTTPLiveRecordManager(downloadConfig, selectedStreams, extractor);
result = await sldm.StartRecordAsync();
}
else if (!livingFlag)
{
// 开始下载
var sdm = new SimpleDownloadManager(downloadConfig, selectedStreams, extractor);
result = await sdm.StartDownloadAsync();
}
else
{
var sldm = new SimpleLiveRecordManager2(downloadConfig, selectedStreams, extractor);
result = await sldm.StartRecordAsync();
}
if (result)
{
Logger.InfoMarkUp("[white on green]Done[/]");
}
else
{
Logger.ErrorMarkUp("[white on red]Failed[/]");
Environment.ExitCode = 1;
}
}
private static async Task WriteRawFilesAsync(MyOption option, StreamExtractor extractor, string tmpDir)
{
// 写出json文件
if (option.WriteMetaJson)
{
if (!Directory.Exists(tmpDir)) Directory.CreateDirectory(tmpDir);
Logger.Warn(ResString.writeJson);
foreach (var item in extractor.RawFiles)
{
var file = Path.Combine(tmpDir, item.Key);
if (!File.Exists(file)) await File.WriteAllTextAsync(file, item.Value, Encoding.UTF8);
}
}
}
static async Task CheckUpdateAsync()
{
try
{
var ver = System.Reflection.Assembly.GetExecutingAssembly().GetName().Version!;
string nowVer = $"v{ver.Major}.{ver.Minor}.{ver.Build}";
string redirctUrl = await Get302Async("https://github.com/nilaoda/N_m3u8DL-RE/releases/latest");
string latestVer = redirctUrl.Replace("https://github.com/nilaoda/N_m3u8DL-RE/releases/tag/", "");
if (!latestVer.StartsWith(nowVer) && !latestVer.StartsWith("https"))
{
Console.Title = $"{ResString.newVersionFound} {latestVer}";
Logger.InfoMarkUp($"[cyan]{ResString.newVersionFound}[/] [red]{latestVer}[/]");
}
}
catch (Exception)
{
;
}
}
// 重定向
static async Task<string> Get302Async(string url)
{
// this allows you to set the settings so that we can get the redirect url
var handler = new HttpClientHandler
{
AllowAutoRedirect = false
};
var redirectedUrl = "";
using var client = new HttpClient(handler);
using var response = await client.GetAsync(url);
using var content = response.Content;
// ... Read the response to see if we have the redirected url
if (response.StatusCode != HttpStatusCode.Found) return redirectedUrl;
var headers = response.Headers;
if (headers.Location != null)
{
redirectedUrl = headers.Location.AbsoluteUri;
}
return redirectedUrl;
}
}

143
src/N_m3u8DL-RE/Util/DownloadUtil.cs Normal file
View File

@ -0,0 +1,143 @@
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Common.Util;
using N_m3u8DL_RE.Entity;
using System.Net.Http.Headers;
namespace N_m3u8DL_RE.Util;
internal static class DownloadUtil
{
private static readonly HttpClient AppHttpClient = HTTPUtil.AppHttpClient;
private static async Task<DownloadResult> CopyFileAsync(string sourceFile, string path, SpeedContainer speedContainer, long? fromPosition = null, long? toPosition = null)
{
using var inputStream = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.Read);
using var outputStream = new FileStream(path, FileMode.OpenOrCreate);
inputStream.Seek(fromPosition ?? 0L, SeekOrigin.Begin);
var expect = (toPosition ?? inputStream.Length) - inputStream.Position + 1;
if (expect == inputStream.Length + 1)
{
await inputStream.CopyToAsync(outputStream);
speedContainer.Add(inputStream.Length);
}
else
{
var buffer = new byte[expect];
_ = await inputStream.ReadAsync(buffer);
await outputStream.WriteAsync(buffer);
speedContainer.Add(buffer.Length);
}
return new DownloadResult()
{
ActualContentLength = outputStream.Length,
ActualFilePath = path
};
}
public static async Task<DownloadResult> DownloadToFileAsync(string url, string path, SpeedContainer speedContainer, CancellationTokenSource cancellationTokenSource, Dictionary<string, string>? headers = null, long? fromPosition = null, long? toPosition = null)
{
Logger.Debug(ResString.fetch + url);
if (url.StartsWith("file:"))
{
var file = new Uri(url).LocalPath;
return await CopyFileAsync(file, path, speedContainer, fromPosition, toPosition);
}
if (url.StartsWith("base64://"))
{
var bytes = Convert.FromBase64String(url[9..]);
await File.WriteAllBytesAsync(path, bytes);
return new DownloadResult()
{
ActualContentLength = bytes.Length,
ActualFilePath = path,
};
}
if (url.StartsWith("hex://"))
{
var bytes = HexUtil.HexToBytes(url[6..]);
await File.WriteAllBytesAsync(path, bytes);
return new DownloadResult()
{
ActualContentLength = bytes.Length,
ActualFilePath = path,
};
}
using var request = new HttpRequestMessage(HttpMethod.Get, new Uri(url));
if (fromPosition != null || toPosition != null)
request.Headers.Range = new(fromPosition, toPosition);
if (headers != null)
{
foreach (var item in headers)
{
request.Headers.TryAddWithoutValidation(item.Key, item.Value);
}
}
Logger.Debug(request.Headers.ToString());
try
{
using var response = await AppHttpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationTokenSource.Token);
if (((int)response.StatusCode).ToString().StartsWith("30"))
{
HttpResponseHeaders respHeaders = response.Headers;
Logger.Debug(respHeaders.ToString());
if (respHeaders.Location != null)
{
var redirectedUrl = "";
if (!respHeaders.Location.IsAbsoluteUri)
{
Uri uri1 = new Uri(url);
Uri uri2 = new Uri(uri1, respHeaders.Location);
redirectedUrl = uri2.ToString();
}
else
{
redirectedUrl = respHeaders.Location.AbsoluteUri;
}
return await DownloadToFileAsync(redirectedUrl, path, speedContainer, cancellationTokenSource, headers, fromPosition, toPosition);
}
}
response.EnsureSuccessStatusCode();
var contentLength = response.Content.Headers.ContentLength;
if (speedContainer.SingleSegment) speedContainer.ResponseLength = contentLength;
using var stream = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.None);
using var responseStream = await response.Content.ReadAsStreamAsync(cancellationTokenSource.Token);
var buffer = new byte[16 * 1024];
var size = 0;
size = await responseStream.ReadAsync(buffer, cancellationTokenSource.Token);
speedContainer.Add(size);
await stream.WriteAsync(buffer.AsMemory(0, size));
// 检测imageHeader
bool imageHeader = ImageHeaderUtil.IsImageHeader(buffer);
// 检测GZip（For DDP Audio）
bool gZipHeader = buffer.Length > 2 && buffer[0] == 0x1f && buffer[1] == 0x8b;
while ((size = await responseStream.ReadAsync(buffer, cancellationTokenSource.Token)) > 0)
{
speedContainer.Add(size);
await stream.WriteAsync(buffer.AsMemory(0, size));
// 限速策略
while (speedContainer.Downloaded > speedContainer.SpeedLimit)
{
await Task.Delay(1);
}
}
return new DownloadResult()
{
ActualContentLength = stream.Length,
RespContentLength = contentLength,
ActualFilePath = path,
ImageHeader= imageHeader,
GzipHeader = gZipHeader
};
}
catch (OperationCanceledException oce) when (oce.CancellationToken == cancellationTokenSource.Token)
{
speedContainer.ResetLowSpeedCount();
throw new Exception("Download speed too slow!");
}
}
}

273
src/N_m3u8DL-RE/Util/FilterUtil.cs Normal file
View File

@ -0,0 +1,273 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Enum;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using N_m3u8DL_RE.Entity;
using Spectre.Console;
using System.Text.RegularExpressions;
namespace N_m3u8DL_RE.Util;
public static class FilterUtil
{
public static List<StreamSpec> DoFilterKeep(IEnumerable<StreamSpec> lists, StreamFilter? filter)
{
if (filter == null) return [];
var inputs = lists.Where(_ => true);
if (filter.GroupIdReg != null)
inputs = inputs.Where(i => i.GroupId != null && filter.GroupIdReg.IsMatch(i.GroupId));
if (filter.LanguageReg != null)
inputs = inputs.Where(i => i.Language != null && filter.LanguageReg.IsMatch(i.Language));
if (filter.NameReg != null)
inputs = inputs.Where(i => i.Name != null && filter.NameReg.IsMatch(i.Name));
if (filter.CodecsReg != null)
inputs = inputs.Where(i => i.Codecs != null && filter.CodecsReg.IsMatch(i.Codecs));
if (filter.ResolutionReg != null)
inputs = inputs.Where(i => i.Resolution != null && filter.ResolutionReg.IsMatch(i.Resolution));
if (filter.FrameRateReg != null)
inputs = inputs.Where(i => i.FrameRate != null && filter.FrameRateReg.IsMatch($"{i.FrameRate}"));
if (filter.ChannelsReg != null)
inputs = inputs.Where(i => i.Channels != null && filter.ChannelsReg.IsMatch(i.Channels));
if (filter.VideoRangeReg != null)
inputs = inputs.Where(i => i.VideoRange != null && filter.VideoRangeReg.IsMatch(i.VideoRange));
if (filter.UrlReg != null)
inputs = inputs.Where(i => i.Url != null && filter.UrlReg.IsMatch(i.Url));
if (filter.SegmentsMaxCount != null && inputs.All(i => i.SegmentsCount > 0))
inputs = inputs.Where(i => i.SegmentsCount < filter.SegmentsMaxCount);
if (filter.SegmentsMinCount != null && inputs.All(i => i.SegmentsCount > 0))
inputs = inputs.Where(i => i.SegmentsCount > filter.SegmentsMinCount);
if (filter.PlaylistMinDur != null)
inputs = inputs.Where(i => i.Playlist?.TotalDuration > filter.PlaylistMinDur);
if (filter.PlaylistMaxDur != null)
inputs = inputs.Where(i => i.Playlist?.TotalDuration < filter.PlaylistMaxDur);
if (filter.BandwidthMin != null)
inputs = inputs.Where(i => i.Bandwidth >= filter.BandwidthMin);
if (filter.BandwidthMax != null)
inputs = inputs.Where(i => i.Bandwidth <= filter.BandwidthMax);
if (filter.Role.HasValue)
inputs = inputs.Where(i => i.Role == filter.Role);
var bestNumberStr = filter.For.Replace("best", "");
var worstNumberStr = filter.For.Replace("worst", "");
if (filter.For == "best" && inputs.Any())
inputs = inputs.Take(1).ToList();
else if (filter.For == "worst" && inputs.Any())
inputs = inputs.TakeLast(1).ToList();
else if (int.TryParse(bestNumberStr, out int bestNumber) && inputs.Any())
inputs = inputs.Take(bestNumber).ToList();
else if (int.TryParse(worstNumberStr, out int worstNumber) && inputs.Any())
inputs = inputs.TakeLast(worstNumber).ToList();
return inputs.ToList();
}
public static List<StreamSpec> DoFilterDrop(IEnumerable<StreamSpec> lists, StreamFilter? filter)
{
if (filter == null) return [..lists];
var inputs = lists.Where(_ => true);
var selected = DoFilterKeep(lists, filter);
inputs = inputs.Where(i => selected.All(s => s.ToString() != i.ToString()));
return inputs.ToList();
}
public static List<StreamSpec> SelectStreams(IEnumerable<StreamSpec> lists)
{
var streamSpecs = lists.ToList();
if (streamSpecs.Count == 1)
return [..streamSpecs];
// 基本流
var basicStreams = streamSpecs.Where(x => x.MediaType == null).ToList();
// 可选音频轨道
var audios = streamSpecs.Where(x => x.MediaType == MediaType.AUDIO).ToList();
// 可选字幕轨道
var subs = streamSpecs.Where(x => x.MediaType == MediaType.SUBTITLES).ToList();
var prompt = new MultiSelectionPrompt<StreamSpec>()
.Title(ResString.promptTitle)
.UseConverter(x =>
{
if (x.Name != null && x.Name.StartsWith("__"))
return $"[darkslategray1]{x.Name[2..]}[/]";
return x.ToString().EscapeMarkup().RemoveMarkup();
})
.Required()
.PageSize(10)
.MoreChoicesText(ResString.promptChoiceText)
.InstructionsText(ResString.promptInfo)
;
// 默认选中第一个
var first = streamSpecs.First();
prompt.Select(first);
if (basicStreams.Count != 0)
{
prompt.AddChoiceGroup(new StreamSpec() { Name = "__Basic" }, basicStreams);
}
if (audios.Count != 0)
{
prompt.AddChoiceGroup(new StreamSpec() { Name = "__Audio" }, audios);
// 默认音轨
if (first.AudioId != null)
{
prompt.Select(audios.First(a => a.GroupId == first.AudioId));
}
}
if (subs.Count != 0)
{
prompt.AddChoiceGroup(new StreamSpec() { Name = "__Subtitle" }, subs);
// 默认字幕轨
if (first.SubtitleId != null)
{
prompt.Select(subs.First(s => s.GroupId == first.SubtitleId));
}
}
// 如果此时还是没有选中任何流，自动选择一个
prompt.Select(basicStreams.Concat(audios).Concat(subs).First());
// 多选
var selectedStreams = CustomAnsiConsole.Console.Prompt(prompt);
return selectedStreams;
}
/// <summary>
/// 直播使用。对齐各个轨道的起始。
/// </summary>
/// <param name="selectedSteams"></param>
/// <param name="takeLastCount"></param>
public static void SyncStreams(List<StreamSpec> selectedSteams, int takeLastCount = 15)
{
// 通过Date同步
if (selectedSteams.All(x => x.Playlist!.MediaParts[0].MediaSegments.All(x => x.DateTime != null)))
{
var minDate = selectedSteams.Max(s => s.Playlist!.MediaParts[0].MediaSegments.Min(s => s.DateTime))!;
foreach (var item in selectedSteams)
{
foreach (var part in item.Playlist!.MediaParts)
{
// 秒级同步 忽略毫秒
part.MediaSegments = part.MediaSegments.Where(s => s.DateTime!.Value.Ticks / TimeSpan.TicksPerSecond >= minDate.Value.Ticks / TimeSpan.TicksPerSecond).ToList();
}
}
}
else // 通过index同步
{
var minIndex = selectedSteams.Max(s => s.Playlist!.MediaParts[0].MediaSegments.Min(s => s.Index));
foreach (var item in selectedSteams)
{
foreach (var part in item.Playlist!.MediaParts)
{
part.MediaSegments = part.MediaSegments.Where(s => s.Index >= minIndex).ToList();
}
}
}
// 取最新的N个分片
if (selectedSteams.Any(x => x.Playlist!.MediaParts[0].MediaSegments.Count > takeLastCount))
{
var skipCount = selectedSteams.Min(x => x.Playlist!.MediaParts[0].MediaSegments.Count) - takeLastCount + 1;
if (skipCount < 0) skipCount = 0;
foreach (var item in selectedSteams)
{
foreach (var part in item.Playlist!.MediaParts)
{
part.MediaSegments = part.MediaSegments.Skip(skipCount).ToList();
}
}
}
}
/// <summary>
/// 应用用户自定义的分片范围
/// </summary>
/// <param name="selectedSteams"></param>
/// <param name="customRange"></param>
public static void ApplyCustomRange(List<StreamSpec> selectedSteams, CustomRange? customRange)
{
if (customRange == null) return;
Logger.InfoMarkUp($"{ResString.customRangeFound}[Cyan underline]{customRange.InputStr}[/]");
Logger.WarnMarkUp($"[darkorange3_1]{ResString.customRangeWarn}[/]");
var filterByIndex = customRange is { StartSegIndex: not null, EndSegIndex: not null };
var filterByTime = customRange is { StartSec: not null, EndSec: not null };
if (!filterByIndex && !filterByTime)
{
Logger.ErrorMarkUp(ResString.customRangeInvalid);
return;
}
foreach (var stream in selectedSteams)
{
var skippedDur = 0d;
if (stream.Playlist == null) continue;
foreach (var part in stream.Playlist.MediaParts)
{
List<MediaSegment> newSegments;
if (filterByIndex)
newSegments = part.MediaSegments.Where(seg => seg.Index >= customRange.StartSegIndex && seg.Index <= customRange.EndSegIndex).ToList();
else
newSegments = part.MediaSegments.Where(seg => stream.Playlist.MediaParts.SelectMany(p => p.MediaSegments).Where(x => x.Index < seg.Index).Sum(x => x.Duration) >= customRange.StartSec
&& stream.Playlist.MediaParts.SelectMany(p => p.MediaSegments).Where(x => x.Index < seg.Index).Sum(x => x.Duration) <= customRange.EndSec).ToList();
if (newSegments.Count > 0)
skippedDur += part.MediaSegments.Where(seg => seg.Index < newSegments.First().Index).Sum(x => x.Duration);
part.MediaSegments = newSegments;
}
stream.SkippedDuration = skippedDur;
}
}
/// <summary>
/// 根据用户输入，清除广告分片
/// </summary>
/// <param name="selectedSteams"></param>
/// <param name="keywords"></param>
public static void CleanAd(List<StreamSpec> selectedSteams, string[]? keywords)
{
if (keywords == null) return;
var regList = keywords.Select(s => new Regex(s)).ToList();
foreach ( var reg in regList)
{
Logger.InfoMarkUp($"{ResString.customAdKeywordsFound}[Cyan underline]{reg}[/]");
}
foreach (var stream in selectedSteams)
{
if (stream.Playlist == null) continue;
var countBefore = stream.SegmentsCount;
foreach (var part in stream.Playlist.MediaParts)
{
// 没有找到广告分片
if (part.MediaSegments.All(x => regList.All(reg => !reg.IsMatch(x.Url))))
{
continue;
}
// 找到广告分片 清理
part.MediaSegments = part.MediaSegments.Where(x => regList.All(reg => !reg.IsMatch(x.Url))).ToList();
}
// 清理已经为空的 part
stream.Playlist.MediaParts = stream.Playlist.MediaParts.Where(x => x.MediaSegments.Count > 0).ToList();
var countAfter = stream.SegmentsCount;
if (countBefore != countAfter)
{
Logger.WarnMarkUp("[grey]{} segments => {} segments[/]", countBefore, countAfter);
}
}
}
}

81
src/N_m3u8DL-RE/Util/ImageHeaderUtil.cs Normal file
View File

@ -0,0 +1,81 @@
namespace N_m3u8DL_RE.Util;
internal static class ImageHeaderUtil
{
public static bool IsImageHeader(byte[] bArr)
{
var size = bArr.Length;
// PNG HEADER检测
if (size > 3 && 137 == bArr[0] && 80 == bArr[1] && 78 == bArr[2] && 71 == bArr[3])
return true;
// GIF HEADER检测
if (size > 3 && 0x47 == bArr[0] && 0x49 == bArr[1] && 0x46 == bArr[2] && 0x38 == bArr[3])
return true;
// BMP HEADER检测
if (size > 10 && 0x42 == bArr[0] && 0x4D == bArr[1] && 0x00 == bArr[5] && 0x00 == bArr[6] && 0x00 == bArr[7] && 0x00 == bArr[8])
return true;
// JPEG HEADER检测
if (size > 3 && 0xFF == bArr[0] && 0xD8 == bArr[1] && 0xFF == bArr[2])
return true;
return false;
}
public static async Task ProcessAsync(string sourcePath)
{
var sourceData = await File.ReadAllBytesAsync(sourcePath);
// PNG HEADER
if (137 == sourceData[0] && 80 == sourceData[1] && 78 == sourceData[2] && 71 == sourceData[3])
{
if (sourceData.Length > 120 && 137 == sourceData[0] && 80 == sourceData[1] && 78 == sourceData[2] && 71 == sourceData[3] && 96 == sourceData[118] && 130 == sourceData[119])
sourceData = sourceData[120..];
else if (sourceData.Length > 6102 && 137 == sourceData[0] && 80 == sourceData[1] && 78 == sourceData[2] && 71 == sourceData[3] && 96 == sourceData[6100] && 130 == sourceData[6101])
sourceData = sourceData[6102..];
else if (sourceData.Length > 69 && 137 == sourceData[0] && 80 == sourceData[1] && 78 == sourceData[2] && 71 == sourceData[3] && 96 == sourceData[67] && 130 == sourceData[68])
sourceData = sourceData[69..];
else if (sourceData.Length > 771 && 137 == sourceData[0] && 80 == sourceData[1] && 78 == sourceData[2] && 71 == sourceData[3] && 96 == sourceData[769] && 130 == sourceData[770])
sourceData = sourceData[771..];
else
{
// 手动查询结尾标记 0x47 出现两次
int skip = 0;
for (int i = 4; i < sourceData.Length - 188 * 2 - 4; i++)
{
if (sourceData[i] == 0x47 && sourceData[i + 188] == 0x47 && sourceData[i + 188 + 188] == 0x47)
{
skip = i;
break;
}
}
sourceData = sourceData[skip..];
}
}
// GIF HEADER
else if (0x47 == sourceData[0] && 0x49 == sourceData[1] && 0x46 == sourceData[2] && 0x38 == sourceData[3])
{
sourceData = sourceData[42..];
}
// BMP HEADER
else if (0x42 == sourceData[0] && 0x4D == sourceData[1] && 0x00 == sourceData[5] && 0x00 == sourceData[6] && 0x00 == sourceData[7] && 0x00 == sourceData[8])
{
sourceData = sourceData[0x3E..];
}
// JPEG HEADER检测
else if (0xFF == sourceData[0] && 0xD8 == sourceData[1] && 0xFF == sourceData[2])
{
// 手动查询结尾标记 0x47 出现两次
int skip = 0;
for (int i = 4; i < sourceData.Length - 188 * 2 - 4; i++)
{
if (sourceData[i] == 0x47 && sourceData[i + 188] == 0x47 && sourceData[i + 188 + 188] == 0x47)
{
skip = i;
break;
}
}
sourceData = sourceData[skip..];
}
await File.WriteAllBytesAsync(sourcePath, sourceData);
}
}

531
src/N_m3u8DL-RE/Util/LanguageCodeUtil.cs Normal file
View File

@ -0,0 +1,531 @@
using N_m3u8DL_RE.Entity;
namespace N_m3u8DL_RE.Util;
internal class Language(string extendCode, string code, string desc, string descA)
{
public readonly string Code = code;
public readonly string ExtendCode = extendCode;
public readonly string Description = desc;
public readonly string DescriptionAudio = descA;
}
internal static class LanguageCodeUtil
{
private static readonly List<Language> ALL_LANGS = @"
default;und;default;default
af;afr;Afrikaans;Afrikaans
af-ZA;afr;Afrikaans (South Africa);Afrikaans (South Africa)
am;amh;Amharic;Amharic
am-ET;amh;Amharic (Ethiopia);Amharic (Ethiopia)
ar;ara;Arabic;Arabic
ar-SA;ara;Arabic (Saudi Arabia);Arabic (Saudi Arabia)
ar-IQ;ara;Arabic (Iraq);Arabic (Iraq)
ar-EG;ara;Arabic (Egypt);Arabic (Egypt)
ar-LY;ara;Arabic (Libya);Arabic (Libya)
ar-DZ;ara;Arabic (Algeria);Arabic (Algeria)
ar-MA;ara;Arabic (Morocco);Arabic (Morocco)
ar-TN;ara;Arabic (Tunisia);Arabic (Tunisia)
ar-OM;ara;Arabic (Oman);Arabic (Oman)
ar-YE;ara;Arabic (Yemen);Arabic (Yemen)
ar-SY;ara;Arabic (Syria);Arabic (Syria)
ar-JO;ara;Arabic (Jordan);Arabic (Jordan)
ar-LB;ara;Arabic (Lebanon);Arabic (Lebanon)
ar-KW;ara;Arabic (Kuwait);Arabic (Kuwait)
ar-AE;ara;Arabic (United Arab Emirates);Arabic (United Arab Emirates)
ar-BH;ara;Arabic (Bahrain);Arabic (Bahrain)
ar-QA;ara;Arabic (Qatar);Arabic (Qatar)
as;asm;Assamese;Assamese
as-IN;asm;Assamese (India);Assamese (India)
az;aze;Azerbaijani;Azerbaijani
az-Latn-AZ;aze;Azerbaijani (Latin, Azerbaijan);Azerbaijani (Latin, Azerbaijan)
az-Cyrl-AZ;aze;Azerbaijani (Cyrillic, Azerbaijan);Azerbaijani (Cyrillic, Azerbaijan)
az-Cyrl;aze;Azerbaijani (Cyrillic);Azerbaijani (Cyrillic)
az-Latn;aze;Azerbaijani (Latin);Azerbaijani (Latin)
be;bel;Belarusian;Belarusian
be-BY;bel;Belarusian (Belarus);Belarusian (Belarus)
bg;bul;Bulgarian;Bulgarian
bg-BG;bul;Bulgarian (Bulgaria);Bulgarian (Bulgaria)
bn;ben;Bangla;Bangla
bn-IN;ben;Bangla (India);Bangla (India)
bn-BD;ben;Bangla (Bangladesh);Bangla (Bangladesh)
bo;bod;Tibetan;Tibetan
bo-CN;bod;Tibetan (China);Tibetan (China)
br;bre;Breton;Breton
br-FR;bre;Breton (France);Breton (France)
bs-Latn-BA;bos;Bosnian (Latin, Bosnia & Herzegovina);Bosnian (Latin, Bosnia & Herzegovina)
bs-Cyrl-BA;bos;Bosnian (Cyrillic, Bosnia & Herzegovina);Bosnian (Cyrillic, Bosnia & Herzegovina)
bs-Cyrl;bos;Bosnian (Cyrillic);Bosnian (Cyrillic)
bs-Latn;bos;Bosnian (Latin);Bosnian (Latin)
bs;bos;Bosnian;Bosnian
ca;cat;Catalan;Catalan
ca-ES;cat;Catalan (Spain);Catalan (Spain)
ca-ES-valencia;cat;Catalan (Spain);Catalan (Spain)
chr;chr;Cherokee;Cherokee
cs;ces;Czech;Czech
cs-CZ;ces;Czech (Czech Republic);Czech (Czech Republic)
cy;cym;Welsh;Welsh
cy-GB;cym;Welsh (United Kingdom);Welsh (United Kingdom)
da;dan;Danish;Danish
da-DK;dan;Danish (Denmark);Danish (Denmark)
de;deu;German;German
de-DE;deu;German (Germany);German (Germany)
de-CH;deu;German (Switzerland);German (Switzerland)
de-AT;deu;German (Austria);German (Austria)
de-LU;deu;German (Luxembourg);German (Luxembourg)
de-LI;deu;German (Liechtenstein);German (Liechtenstein)
dsb-DE;dsb;Lower Sorbian (Germany);Lower Sorbian (Germany)
dsb;dsb;Lower Sorbian;Lower Sorbian
el;ell;Greek;Greek
el-GR;ell;Greek (Greece);Greek (Greece)
en;eng;English;English
en-US;eng;English (United States);English (United States)
en-GB;eng;English (United Kingdom);English (United Kingdom)
en-AU;eng;English (Australia);English (Australia)
en-CA;eng;English (Canada);English (Canada)
en-NZ;eng;English (New Zealand);English (New Zealand)
en-IE;eng;English (Ireland);English (Ireland)
en-ZA;eng;English (South Africa);English (South Africa)
en-JM;eng;English (Jamaica);English (Jamaica)
en-BZ;eng;English (Belize);English (Belize)
en-TT;eng;English (Trinidad & Tobago);English (Trinidad & Tobago)
en-ZW;eng;English (Zimbabwe);English (Zimbabwe)
en-PH;eng;English (Philippines);English (Philippines)
en-HK;eng;English (Hong Kong SAR China);English (Hong Kong SAR China)
en-IN;eng;English (India);English (India)
en-MY;eng;English (Malaysia);English (Malaysia)
en-SG;eng;English (Singapore);English (Singapore)
es;spa;Spanish;Spanish
es-MX;spa;Spanish (Mexico);Spanish (Mexico)
es-ES;spa;Spanish (Spain);Spanish (Spain)
es-GT;spa;Spanish (Guatemala);Spanish (Guatemala)
es-CR;spa;Spanish (Costa Rica);Spanish (Costa Rica)
es-PA;spa;Spanish (Panama);Spanish (Panama)
es-DO;spa;Spanish (Dominican Republic);Spanish (Dominican Republic)
es-VE;spa;Spanish (Venezuela);Spanish (Venezuela)
es-CO;spa;Spanish (Colombia);Spanish (Colombia)
es-PE;spa;Spanish (Peru);Spanish (Peru)
es-AR;spa;Spanish (Argentina);Spanish (Argentina)
es-EC;spa;Spanish (Ecuador);Spanish (Ecuador)
es-CL;spa;Spanish (Chile);Spanish (Chile)
es-UY;spa;Spanish (Uruguay);Spanish (Uruguay)
es-PY;spa;Spanish (Paraguay);Spanish (Paraguay)
es-BO;spa;Spanish (Bolivia);Spanish (Bolivia)
es-SV;spa;Spanish (El Salvador);Spanish (El Salvador)
es-HN;spa;Spanish (Honduras);Spanish (Honduras)
es-NI;spa;Spanish (Nicaragua);Spanish (Nicaragua)
es-PR;spa;Spanish (Puerto Rico);Spanish (Puerto Rico)
es-US;spa;Spanish (United States);Spanish (United States)
es-CU;spa;Spanish (Cuba);Spanish (Cuba)
et;est;Estonian;Estonian
et-EE;est;Estonian (Estonia);Estonian (Estonia)
eu;eus;Basque;Basque
eu-ES;eus;Basque (Spain);Basque (Spain)
fa;fas;Persian;Persian
fa-IR;fas;Persian (Iran);Persian (Iran)
ff;ful;Fulah;Fulah
fi;fin;Finnish;Finnish
fi-FI;fin;Finnish (Finland);Finnish (Finland)
fil;fil;Filipino;Filipino
fil-PH;fil;Filipino (Philippines);Filipino (Philippines)
fo;fao;Faroese;Faroese
fo-FO;fao;Faroese (Faroe Islands);Faroese (Faroe Islands)
fr;fra;French;French
fr-FR;fra;French (France);French (France)
fr-BE;fra;French (Belgium);French (Belgium)
fr-CA;fra;French (Canada);French (Canada)
fr-CH;fra;French (Switzerland);French (Switzerland)
fr-LU;fra;French (Luxembourg);French (Luxembourg)
fr-MC;fra;French (Monaco);French (Monaco)
fr-RE;fra;French (Réunion);French (Réunion)
fr-CD;fra;French (Congo - Kinshasa);French (Congo - Kinshasa)
fr-SN;fra;French (Senegal);French (Senegal)
fr-CM;fra;French (Cameroon);French (Cameroon)
fr-CI;fra;French (Côte d’Ivoire);French (Côte d’Ivoire)
fr-ML;fra;French (Mali);French (Mali)
fr-MA;fra;French (Morocco);French (Morocco)
fr-HT;fra;French (Haiti);French (Haiti)
fy;fry;Western Frisian;Western Frisian
fy-NL;fry;Western Frisian (Netherlands);Western Frisian (Netherlands)
ga;gle;Irish;Irish
ga-IE;gle;Irish (Ireland);Irish (Ireland)
gd;gla;Scottish Gaelic;Scottish Gaelic
gd-GB;gla;Scottish Gaelic (United Kingdom);Scottish Gaelic (United Kingdom)
gl;glg;Galician;Galician
gl-ES;glg;Galician (Spain);Galician (Spain)
gsw;gsw;Swiss German;Swiss German
gsw-FR;gsw;Swiss German (France);Swiss German (France)
gu;guj;Gujarati;Gujarati
gu-IN;guj;Gujarati (India);Gujarati (India)
ha;hau;Hausa;Hausa
ha-Latn-NG;hau;Hausa (Latin, Nigeria);Hausa (Latin, Nigeria)
ha-Latn;hau;Hausa (Latin);Hausa (Latin)
haw;haw;Hawaiian;Hawaiian
haw-US;haw;Hawaiian (United States);Hawaiian (United States)
he;heb;Hebrew;Hebrew
he-IL;heb;Hebrew (Israel);Hebrew (Israel)
hi;hin;Hindi;Hindi
hi-IN;hin;Hindi (India);Hindi (India)
hr;hrv;Croatian;Croatian
hr-HR;hrv;Croatian (Croatia);Croatian (Croatia)
hr-BA;hrv;Croatian (Bosnia & Herzegovina);Croatian (Bosnia & Herzegovina)
hsb;hsb;Upper Sorbian;Upper Sorbian
hsb-DE;hsb;Upper Sorbian (Germany);Upper Sorbian (Germany)
hu;hun;Hungarian;Hungarian
hu-HU;hun;Hungarian (Hungary);Hungarian (Hungary)
hy;hye;Armenian;Armenian
hy-AM;hye;Armenian (Armenia);Armenian (Armenia)
id;ind;Indonesian;Indonesian
id-ID;ind;Indonesian (Indonesia);Indonesian (Indonesia)
ig;ibo;Igbo;Igbo
ig-NG;ibo;Igbo (Nigeria);Igbo (Nigeria)
ii;iii;Sichuan Yi;Sichuan Yi
ii-CN;iii;Sichuan Yi (China);Sichuan Yi (China)
is;isl;Icelandic;Icelandic
is-IS;isl;Icelandic (Iceland);Icelandic (Iceland)
it;ita;Italian;Italian
it-IT;ita;Italian (Italy);Italian (Italy)
it-CH;ita;Italian (Switzerland);Italian (Switzerland)
ja;jpn;Japanese;Japanese
ja-JP;jpn;Japanese (Japan);Japanese (Japan)
ka;kat;Georgian;Georgian
ka-GE;kat;Georgian (Georgia);Georgian (Georgia)
kk;kaz;Kazakh;Kazakh
kk-KZ;kaz;Kazakh (Kazakhstan);Kazakh (Kazakhstan)
kl;kal;Kalaallisut;Kalaallisut
kl-GL;kal;Kalaallisut (Greenland);Kalaallisut (Greenland)
km;khm;Khmer;Khmer
km-KH;khm;Khmer (Cambodia);Khmer (Cambodia)
kn;kan;Kannada;Kannada
kn-IN;kan;Kannada (India);Kannada (India)
ko;kor;Korean;Korean
ko-KR;kor;Korean (South Korea);Korean (South Korea)
kok;kok;Konkani;Konkani
kok-IN;kok;Konkani (India);Konkani (India)
ky;kir;Kyrgyz;Kyrgyz
ky-KG;kir;Kyrgyz (Kyrgyzstan);Kyrgyz (Kyrgyzstan)
lb;ltz;Luxembourgish;Luxembourgish
lb-LU;ltz;Luxembourgish (Luxembourg);Luxembourgish (Luxembourg)
lo;lao;Lao;Lao
lo-LA;lao;Lao (Laos);Lao (Laos)
lt;lit;Lithuanian;Lithuanian
lt-LT;lit;Lithuanian (Lithuania);Lithuanian (Lithuania)
lv;lav;Latvian;Latvian
lv-LV;lav;Latvian (Latvia);Latvian (Latvia)
mk;mkd;Macedonian;Macedonian
mk-MK;mkd;Macedonian (Macedonia);Macedonian (Macedonia)
ml;mal;Malayalam;Malayalam
ml-IN;mal;Malayalam (India);Malayalam (India)
mn;mon;Mongolian;Mongolian
mn-MN;mon;Mongolian (Mongolia);Mongolian (Mongolia)
mn-Cyrl;mon;Mongolian (Cyrillic);Mongolian (Cyrillic)
mr;mar;Marathi;Marathi
mr-IN;mar;Marathi (India);Marathi (India)
ms;msa;Malay;Malay
ms-MY;msa;Malay (Malaysia);Malay (Malaysia)
ms-BN;msa;Malay (Brunei);Malay (Brunei)
mt;mlt;Maltese;Maltese
mt-MT;mlt;Maltese (Malta);Maltese (Malta)
my;mya;Burmese;Burmese
my-MM;mya;Burmese (Myanmar (Burma));Burmese (Myanmar (Burma))
no;nob;Norwegian;Norwegian
nb-NO;nob;Norwegian Bokmål (Norway);Norwegian Bokmål (Norway)
nb;nob;Norwegian Bokmål;Norwegian Bokmål
ne;nep;Nepali;Nepali
ne-NP;nep;Nepali (Nepal);Nepali (Nepal)
ne-IN;nep;Nepali (India);Nepali (India)
nl;nld;Dutch;Dutch
nl-NL;nld;Dutch (Netherlands);Dutch (Netherlands)
nl-BE;nld;Dutch (Belgium);Dutch (Belgium)
nn-NO;nno;Norwegian Nynorsk (Norway);Norwegian Nynorsk (Norway)
nn;nno;Norwegian Nynorsk;Norwegian Nynorsk
nso;nso;Northern Sotho;Northern Sotho
nso-ZA;nso;Northern Sotho (South Africa);Northern Sotho (South Africa)
om;orm;Oromo;Oromo
om-ET;orm;Oromo (Ethiopia);Oromo (Ethiopia)
or;ori;Odia;Odia
or-IN;ori;Odia (India);Odia (India)
pa;pan;Punjabi;Punjabi
pa-Arab-PK;pan;Punjabi (Arabic, Pakistan);Punjabi (Arabic, Pakistan)
pa-Arab;pan;Punjabi (Arabic);Punjabi (Arabic)
pl;pol;Polish;Polish
pl-PL;pol;Polish (Poland);Polish (Poland)
ps;pus;Pashto;Pashto
ps-AF;pus;Pashto (Afghanistan);Pashto (Afghanistan)
pt;por;Portuguese;Portuguese
pt-BR;por;Portuguese (Brazil);Portuguese (Brazil)
pt-PT;por;Portuguese (Portugal);Portuguese (Portugal)
rm;roh;Romansh;Romansh
rm-CH;roh;Romansh (Switzerland);Romansh (Switzerland)
ro;ron;Romanian;Romanian
ro-RO;ron;Romanian (Romania);Romanian (Romania)
ro-MD;ron;Romanian (Moldova);Romanian (Moldova)
ru;rus;Russian;Russian
ru-RU;rus;Russian (Russia);Russian (Russia)
ru-MD;rus;Russian (Moldova);Russian (Moldova)
rw;kin;Kinyarwanda;Kinyarwanda
rw-RW;kin;Kinyarwanda (Rwanda);Kinyarwanda (Rwanda)
sah;sah;Sakha;Sakha
sah-RU;sah;Sakha (Russia);Sakha (Russia)
se;sme;Northern Sami;Northern Sami
se-NO;sme;Northern Sami (Norway);Northern Sami (Norway)
se-SE;sme;Northern Sami (Sweden);Northern Sami (Sweden)
se-FI;sme;Northern Sami (Finland);Northern Sami (Finland)
si;sin;Sinhala;Sinhala
si-LK;sin;Sinhala (Sri Lanka);Sinhala (Sri Lanka)
sk;slk;Slovak;Slovak
sk-SK;slk;Slovak (Slovakia);Slovak (Slovakia)
sl;slv;Slovenian;Slovenian
sl-SI;slv;Slovenian (Slovenia);Slovenian (Slovenia)
smn-FI;smn;Inari Sami (Finland);Inari Sami (Finland)
smn;smn;Inari Sami;Inari Sami
so;som;Somali;Somali
so-SO;som;Somali (Somalia);Somali (Somalia)
sq;sqi;Albanian;Albanian
sq-AL;sqi;Albanian (Albania);Albanian (Albania)
sr-Latn-BA;srp;Serbian (Latin, Bosnia & Herzegovina);Serbian (Latin, Bosnia & Herzegovina)
sr-Cyrl-BA;srp;Serbian (Cyrillic, Bosnia & Herzegovina);Serbian (Cyrillic, Bosnia & Herzegovina)
sr-Latn-RS;srp;Serbian (Latin, Serbia);Serbian (Latin, Serbia)
sr-Cyrl-RS;srp;Serbian (Cyrillic, Serbia);Serbian (Cyrillic, Serbia)
sr-Latn-ME;srp;Serbian (Latin, Montenegro);Serbian (Latin, Montenegro)
sr-Cyrl-ME;srp;Serbian (Cyrillic, Montenegro);Serbian (Cyrillic, Montenegro)
sr-Cyrl;srp;Serbian (Cyrillic);Serbian (Cyrillic)
sr-Latn;srp;Serbian (Latin);Serbian (Latin)
sr;srp;Serbian;Serbian
st;sot;Southern Sotho;Southern Sotho
st-ZA;sot;Southern Sotho (South Africa);Southern Sotho (South Africa)
sv;swe;Swedish;Swedish
sv-SE;swe;Swedish (Sweden);Swedish (Sweden)
sv-FI;swe;Swedish (Finland);Swedish (Finland)
sw;swa;Swahili;Swahili
sw-KE;swa;Swahili (Kenya);Swahili (Kenya)
ta;tam;Tamil;Tamil
ta-IN;tam;Tamil (India);Tamil (India)
ta-LK;tam;Tamil (Sri Lanka);Tamil (Sri Lanka)
te;tel;Telugu;Telugu
te-IN;tel;Telugu (India);Telugu (India)
tg;tgk;Tajik;Tajik
tg-Cyrl-TJ;tgk;Tajik (Cyrillic, Tajikistan);Tajik (Cyrillic, Tajikistan)
tg-Cyrl;tgk;Tajik (Cyrillic);Tajik (Cyrillic)
th;tha;Thai;Thai
th-TH;tha;Thai (Thailand);Thai (Thailand)
ti;tir;Tigrinya;Tigrinya
ti-ET;tir;Tigrinya (Ethiopia);Tigrinya (Ethiopia)
ti-ER;tir;Tigrinya (Eritrea);Tigrinya (Eritrea)
tk;tuk;Turkmen;Turkmen
tk-TM;tuk;Turkmen (Turkmenistan);Turkmen (Turkmenistan)
tn;tsn;Tswana;Tswana
tn-ZA;tsn;Tswana (South Africa);Tswana (South Africa)
tn-BW;tsn;Tswana (Botswana);Tswana (Botswana)
tr;tur;Turkish;Turkish
tr-TR;tur;Turkish (Turkey);Turkish (Turkey)
ts;tso;Tsonga;Tsonga
ts-ZA;tso;Tsonga (South Africa);Tsonga (South Africa)
tzm;tzm;Central Atlas Tamazight;Central Atlas Tamazight
tzm-Latn;tzm;Central Atlas Tamazight (Latin);Central Atlas Tamazight (Latin)
ug;uig;Uyghur;Uyghur
ug-CN;uig;Uyghur (China);Uyghur (China)
uk;ukr;Ukrainian;Ukrainian
uk-UA;ukr;Ukrainian (Ukraine);Ukrainian (Ukraine)
ur;urd;Urdu;Urdu
ur-PK;urd;Urdu (Pakistan);Urdu (Pakistan)
ur-IN;urd;Urdu (India);Urdu (India)
uz;uzb;Uzbek;Uzbek
uz-Latn-UZ;uzb;Uzbek (Latin, Uzbekistan);Uzbek (Latin, Uzbekistan)
uz-Cyrl-UZ;uzb;Uzbek (Cyrillic, Uzbekistan);Uzbek (Cyrillic, Uzbekistan)
uz-Cyrl;uzb;Uzbek (Cyrillic);Uzbek (Cyrillic)
uz-Latn;uzb;Uzbek (Latin);Uzbek (Latin)
vi;vie;Vietnamese;Vietnamese
vi-VN;vie;Vietnamese (Vietnam);Vietnamese (Vietnam)
xh;xho;Xhosa;Xhosa
xh-ZA;xho;Xhosa (South Africa);Xhosa (South Africa)
yo;yor;Yoruba;Yoruba
yo-NG;yor;Yoruba (Nigeria);Yoruba (Nigeria)
zu;zul;Zulu;Zulu
zu-ZA;zul;Zulu (South Africa);Zulu (South Africa)
zho;chi;中文;中文
chi;chi;中文;中文
chs;chi;中文（简体）;中文
zh-CN;chi;中文（简体）;中文
zh-SG;chi;中文（简体, 新加坡）;中文
zh-MO;chi;中文（繁體, 澳門）;中文
zh-Hans;chi;中文（简体）;中文
zh-Hant;chi;中文（繁體）;中文
zh-TW;chi;中文（繁體, 台灣）;中文
zh-Hant-TW;chi;中文（繁體, 台灣）;中文
zh-HK;chi;中文（繁體, 香港）;中文
zh-Hant-HK;chi;中文（繁體, 香港）;中文
yue;chi;中文（繁體）;粵語
cmn;chi;中文（简体）;普通话
cmn-Hans;chi;中文（简体）;普通话
cmn-Hant;chi;中文（繁體）;普通話
Cantonese;chi;中文;粵語
Mandarin;chi;中文;普通话
Japanese;jpn;日本語;日本語
Korean;kor;한국어;한국어
Vietnamese;vie;Vietnamese;Vietnamese
English;eng;English;English
Thai;tha;Thai;Thai
CN;chi;中文（繁體）;中文
CC;chi;中文（繁體）;中文
CZ;chi;中文（简体）;中文
MA;msa;Melayu;Melayu
"
.Trim().Replace("\r", "").Split('\n').Where(x => !string.IsNullOrWhiteSpace(x)).Select(x =>
{
var arr = x.Trim().Split(';', StringSplitOptions.TrimEntries);
return new Language(arr[0], arr[1], arr[2], arr[3]);
}).ToList();
private static Dictionary<string, string> CODE_MAP = @"
iv;IVL
ar;ara
bg;bul
ca;cat
zh;zho
cs;ces
da;dan
de;deu
el;ell
en;eng
es;spa
fi;fin
fr;fra
he;heb
hu;hun
is;isl
it;ita
ja;jpn
ko;kor
nl;nld
nb;nob
pl;pol
pt;por
rm;roh
ro;ron
ru;rus
hr;hrv
sk;slk
sq;sqi
sv;swe
th;tha
tr;tur
ur;urd
id;ind
uk;ukr
be;bel
sl;slv
et;est
lv;lav
lt;lit
tg;tgk
fa;fas
vi;vie
hy;hye
az;aze
eu;eus
mk;mkd
st;sot
ts;tso
tn;tsn
xh;xho
zu;zul
af;afr
ka;kat
fo;fao
hi;hin
mt;mlt
se;sme
ga;gle
ms;msa
kk;kaz
ky;kir
sw;swa
tk;tuk
uz;uzb
bn;ben
pa;pan
gu;guj
or;ori
ta;tam
te;tel
kn;kan
ml;mal
as;asm
mr;mar
mn;mon
bo;bod
cy;cym
km;khm
lo;lao
my;mya
gl;glg
si;sin
am;amh
ne;nep
fy;fry
ps;pus
ff;ful
ha;hau
yo;yor
lb;ltz
kl;kal
ig;ibo
om;orm
ti;tir
so;som
ii;iii
br;bre
ug;uig
rw;kin
gd;gla
nn;nno
bs;bos
sr;srp
"
.Trim().Replace("\r", "").Split('\n').Where(x => !string.IsNullOrWhiteSpace(x)).Select(x => x.Trim()).ToDictionary(x => x.Split(';').First().Trim(), x => x.Split(';').Last().Trim());
private static string ConvertTwoToThree(string input)
{
return CODE_MAP.GetValueOrDefault(input, input);
}
/// <summary>
/// 转换 ISO 639-1 => ISO 639-2
/// 且当Description为空时将DisplayName写入
/// </summary>
/// <param name="outputFile"></param>
public static void ConvertLangCodeAndDisplayName(OutputFile outputFile)
{
if (string.IsNullOrEmpty(outputFile.LangCode)) return;
var originalLangCode = outputFile.LangCode;
// 先直接查找
var lang = ALL_LANGS.FirstOrDefault(a => a.ExtendCode.Equals(outputFile.LangCode, StringComparison.OrdinalIgnoreCase) || a.Code.Equals(outputFile.LangCode, StringComparison.OrdinalIgnoreCase));
// 处理特殊的扩展语言标记
if (lang == null)
{
// 2位转3位
var l = ConvertTwoToThree(outputFile.LangCode.Split('-').First());
lang = ALL_LANGS.FirstOrDefault(a => a.ExtendCode.Equals(l, StringComparison.OrdinalIgnoreCase) || a.Code.Equals(l, StringComparison.OrdinalIgnoreCase));
}
if (lang != null)
{
outputFile.LangCode = lang.Code;
if (string.IsNullOrEmpty(outputFile.Description))
outputFile.Description = outputFile.MediaType == Common.Enum.MediaType.SUBTITLES ? lang.Description : lang.DescriptionAudio;
}
else
{
outputFile.LangCode = "und"; // 无法识别直接置为und
}
// 无描述，则把LangCode当作描述
if (string.IsNullOrEmpty(outputFile.Description)) outputFile.Description = originalLangCode;
}
}

113
src/N_m3u8DL-RE/Util/LargeSingleFileSplitUtil.cs Normal file
View File

@ -0,0 +1,113 @@
using N_m3u8DL_RE.Common.Entity;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Util;
namespace N_m3u8DL_RE.Util;
internal static class LargeSingleFileSplitUtil
{
class Clip
{
public required int Index;
public required long From;
public required long To;
}
/// <summary>
/// URL大文件切片处理
/// </summary>
/// <param name="segment"></param>
/// <param name="headers"></param>
/// <returns></returns>
public static async Task<List<MediaSegment>?> SplitUrlAsync(MediaSegment segment, Dictionary<string,string> headers)
{
var url = segment.Url;
if (!await CanSplitAsync(url, headers)) return null;
if (segment.StartRange != null) return null;
long fileSize = await GetFileSizeAsync(url, headers);
if (fileSize == 0) return null;
List<Clip> allClips = GetAllClips(url, fileSize);
var splitSegments = new List<MediaSegment>();
foreach (Clip clip in allClips)
{
splitSegments.Add(new MediaSegment()
{
Index = clip.Index,
Url = url,
StartRange = clip.From,
ExpectLength = clip.To == -1 ? null : clip.To - clip.From + 1,
EncryptInfo = segment.EncryptInfo,
});
}
return splitSegments;
}
public static async Task<bool> CanSplitAsync(string url, Dictionary<string, string> headers)
{
try
{
var request = new HttpRequestMessage(HttpMethod.Head, url);
var response = (await HTTPUtil.AppHttpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead)).EnsureSuccessStatusCode();
bool supportsRangeRequests = response.Headers.Contains("Accept-Ranges");
return supportsRangeRequests;
}
catch (Exception ex)
{
Logger.DebugMarkUp(ex.Message);
return false;
}
}
private static async Task<long> GetFileSizeAsync(string url, Dictionary<string, string> headers)
{
using var httpRequestMessage = new HttpRequestMessage();
httpRequestMessage.RequestUri = new(url);
foreach (var header in headers)
{
httpRequestMessage.Headers.TryAddWithoutValidation(header.Key, header.Value);
}
var response = (await HTTPUtil.AppHttpClient.SendAsync(httpRequestMessage, HttpCompletionOption.ResponseHeadersRead)).EnsureSuccessStatusCode();
long totalSizeBytes = response.Content.Headers.ContentLength ?? 0;
return totalSizeBytes;
}
// 此函数主要是切片下载逻辑
private static List<Clip> GetAllClips(string url, long fileSize)
{
List<Clip> clips = [];
int index = 0;
long counter = 0;
int perSize = 10 * 1024 * 1024;
while (fileSize > 0)
{
Clip c = new()
{
Index = index,
From = counter,
To = counter + perSize
};
// 没到最后
if (fileSize - perSize > 0)
{
fileSize -= perSize;
counter += perSize + 1;
index++;
clips.Add(c);
}
// 已到最后
else
{
c.To = -1;
clips.Add(c);
break;
}
}
return clips;
}
}

219
src/N_m3u8DL-RE/Util/MP4DecryptUtil.cs Normal file
View File

@ -0,0 +1,219 @@
using Mp4SubtitleParser;
using N_m3u8DL_RE.Common.Log;
using N_m3u8DL_RE.Common.Resource;
using System.Diagnostics;
using System.Text.RegularExpressions;
using N_m3u8DL_RE.Enum;
namespace N_m3u8DL_RE.Util;
internal static partial class MP4DecryptUtil
{
private static readonly string ZeroKid = "00000000000000000000000000000000";
public static async Task<bool> DecryptAsync(DecryptEngine decryptEngine, string bin, string[]? keys, string source, string dest, string? kid, string init = "", bool isMultiDRM=false)
{
if (keys == null || keys.Length == 0) return false;
var keyPairs = keys.ToList();
string? keyPair = null;
string? trackId = null;
string? tmpEncFile = null;
string? tmpDecFile = null;
string? workDir = null;
if (isMultiDRM)
{
trackId = "1";
}
if (!string.IsNullOrEmpty(kid))
{
var test = keyPairs.Where(k => k.StartsWith(kid)).ToList();
if (test.Count != 0) keyPair = test.First();
}
// Apple
if (kid == ZeroKid)
{
keyPair = keyPairs.First();
trackId = "1";
}
// user only input key, append kid
if (keyPair == null && keyPairs.Count == 1 && !keyPairs.First().Contains(':'))
{
keyPairs = keyPairs.Select(x => $"{kid}:{x}").ToList();
keyPair = keyPairs.First();
}
if (keyPair == null) return false;
// shakaPackager/ffmpeg 无法单独解密init文件
if (source.EndsWith("_init.mp4") && decryptEngine != DecryptEngine.MP4DECRYPT) return false;
string cmd;
var tmpFile = "";
if (decryptEngine == DecryptEngine.SHAKA_PACKAGER)
{
var enc = source;
// shakaPackager 手动构造文件
if (init != "")
{
tmpFile = Path.ChangeExtension(source, ".itmp");
MergeUtil.CombineMultipleFilesIntoSingleFile([init, source], tmpFile);
enc = tmpFile;
}
cmd = $"--quiet --enable_raw_key_decryption input=\"{enc}\",stream=0,output=\"{dest}\" " +
$"--keys {(trackId != null ? $"label={trackId}:" : "")}key_id={(trackId != null ? ZeroKid : kid)}:key={keyPair.Split(':')[1]}";
}
else if (decryptEngine == DecryptEngine.MP4DECRYPT)
{
if (trackId == null)
{
cmd = string.Join(" ", keyPairs.Select(k => $"--key {k}"));
}
else
{
cmd = string.Join(" ", keyPairs.Select(k => $"--key {trackId}:{k.Split(':')[1]}"));
}
// 解决mp4decrypt中文问题 切换到源文件所在目录并改名再解密
workDir = Path.GetDirectoryName(source)!;
tmpEncFile = Path.Combine(workDir, $"{Guid.NewGuid()}{Path.GetExtension(source)}");
tmpDecFile = Path.Combine(workDir, $"{Path.GetFileNameWithoutExtension(tmpEncFile)}_dec{Path.GetExtension(tmpEncFile)}");
File.Move(source, tmpEncFile);
if (init != "")
{
var infoFile = Path.GetDirectoryName(init) == workDir ? Path.GetFileName(init) : init;
cmd += $" --fragments-info \"{infoFile}\" ";
}
cmd += $" \"{Path.GetFileName(tmpEncFile)}\" \"{Path.GetFileName(tmpDecFile)}\"";
}
else
{
var enc = source;
// ffmpeg实时解密 手动构造文件
if (init != "")
{
tmpFile = Path.ChangeExtension(source, ".itmp");
MergeUtil.CombineMultipleFilesIntoSingleFile([init, source], tmpFile);
enc = tmpFile;
}
cmd = $"-loglevel error -nostdin -decryption_key {keyPair.Split(':')[1]} -i \"{enc}\" -c copy \"{dest}\"";
}
var isSuccess = await RunCommandAsync(bin, cmd, workDir);
// mp4decrypt 还原文件改名操作
if (workDir is not null)
{
if (File.Exists(tmpEncFile)) File.Move(tmpEncFile, source);
if (File.Exists(tmpDecFile)) File.Move(tmpDecFile, dest);
}
if (isSuccess)
{
if (tmpFile != "" && File.Exists(tmpFile)) File.Delete(tmpFile);
return true;
}
Logger.Error(ResString.decryptionFailed);
return false;
}
private static async Task<bool> RunCommandAsync(string name, string arg, string? workDir = null)
{
Logger.DebugMarkUp($"FileName: {name}");
Logger.DebugMarkUp($"Arguments: {arg}");
var process = Process.Start(new ProcessStartInfo()
{
FileName = name,
Arguments = arg,
// RedirectStandardOutput = true,
// RedirectStandardError = true,
CreateNoWindow = true,
UseShellExecute = false,
WorkingDirectory = workDir
});
await process!.WaitForExitAsync();
return process.ExitCode == 0;
}
/// <summary>
/// 从文本文件中查询KID的KEY
/// </summary>
/// <param name="file">文本文件</param>
/// <param name="kid">目标KID</param>
/// <returns></returns>
public static async Task<string?> SearchKeyFromFileAsync(string? file, string? kid)
{
try
{
if (string.IsNullOrEmpty(file) || !File.Exists(file) || string.IsNullOrEmpty(kid))
return null;
Logger.InfoMarkUp(ResString.searchKey);
using var stream = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.Read);
using var reader = new StreamReader(stream);
while (await reader.ReadLineAsync() is { } line)
{
if (!line.Trim().StartsWith(kid)) continue;
Logger.InfoMarkUp($"[green]OK[/] [grey]{line.Trim()}[/]");
return line.Trim();
}
}
catch (Exception ex)
{
Logger.ErrorMarkUp(ex.Message);
}
return null;
}
public static ParsedMP4Info GetMP4Info(byte[] data)
{
var info = MP4InitUtil.ReadInit(data);
if (info.Scheme != null) Logger.WarnMarkUp($"[grey]Type: {info.Scheme}[/]");
if (info.PSSH != null) Logger.WarnMarkUp($"[grey]PSSH(WV): {info.PSSH}[/]");
if (info.KID != null) Logger.WarnMarkUp($"[grey]KID: {info.KID}[/]");
return info;
}
public static ParsedMP4Info GetMP4Info(string output)
{
using var fs = File.OpenRead(output);
var header = new byte[1 * 1024 * 1024]; // 1MB
_ = fs.Read(header);
return GetMP4Info(header);
}
public static string? ReadInitShaka(string output, string bin)
{
Regex shakaKeyIdRegex = KidOutputRegex();
// TODO: handle the case that shaka packager actually decrypted (key ID == ZeroKid)
// - stop process
// - remove {output}.tmp.webm
var cmd = $"--quiet --enable_raw_key_decryption input=\"{output}\",stream=0,output=\"{output}.tmp.webm\" " +
$"--keys key_id={ZeroKid}:key={ZeroKid}";
using var p = new Process();
p.StartInfo = new ProcessStartInfo()
{
FileName = bin,
Arguments = cmd,
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false
};
p.Start();
var errorOutput = p.StandardError.ReadToEnd();
p.WaitForExit();
return shakaKeyIdRegex.Match(errorOutput).Groups[1].Value;
}
[GeneratedRegex("Key for key_id=([0-9a-f]+) was not found")]
private static partial Regex KidOutputRegex();
}

92
src/N_m3u8DL-RE/Util/MediainfoUtil.cs Normal file
View File

@ -0,0 +1,92 @@
using N_m3u8DL_RE.Entity;
using System.Diagnostics;
using System.Text.RegularExpressions;
namespace N_m3u8DL_RE.Util;
internal static partial class MediainfoUtil
{
[GeneratedRegex(" Stream #.*")]
private static partial Regex TextRegex();
[GeneratedRegex(@"#0:\d(\[0x\w+?\])")]
private static partial Regex IdRegex();
[GeneratedRegex(": (\\w+): (.*)")]
private static partial Regex TypeRegex();
[GeneratedRegex("(.*?)(,|$)")]
private static partial Regex BaseInfoRegex();
[GeneratedRegex(@" \/ 0x\w+")]
private static partial Regex ReplaceRegex();
[GeneratedRegex(@"\d{2,}x\d+")]
private static partial Regex ResRegex();
[GeneratedRegex(@"\d+ kb\/s")]
private static partial Regex BitrateRegex();
[GeneratedRegex(@"(\d+(\.\d+)?) fps")]
private static partial Regex FpsRegex();
[GeneratedRegex(@"DOVI configuration record.*profile: (\d).*compatibility id: (\d)")]
private static partial Regex DoViRegex();
[GeneratedRegex(@"Duration.*?start: (\d+\.?\d{0,3})")]
private static partial Regex StartRegex();
public static async Task<List<Mediainfo>> ReadInfoAsync(string binary, string file)
{
var result = new List<Mediainfo>();
if (string.IsNullOrEmpty(file) || !File.Exists(file)) return result;
string cmd = "-hide_banner -i \"" + file + "\"";
var p = Process.Start(new ProcessStartInfo()
{
FileName = binary,
Arguments = cmd,
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false
})!;
var output = await p.StandardError.ReadToEndAsync();
await p.WaitForExitAsync();
foreach (Match stream in TextRegex().Matches(output))
{
var info = new Mediainfo()
{
Text = TypeRegex().Match(stream.Value).Groups[2].Value.TrimEnd(),
Id = IdRegex().Match(stream.Value).Groups[1].Value,
Type = TypeRegex().Match(stream.Value).Groups[1].Value,
};
info.Resolution = ResRegex().Match(info.Text).Value;
info.Bitrate = BitrateRegex().Match(info.Text).Value;
info.Fps = FpsRegex().Match(info.Text).Value;
info.BaseInfo = BaseInfoRegex().Match(info.Text).Groups[1].Value;
info.BaseInfo = ReplaceRegex().Replace(info.BaseInfo, "");
info.HDR = info.Text.Contains("/bt2020/");
if (info.BaseInfo.Contains("dvhe")
|| info.BaseInfo.Contains("dvh1")
|| info.BaseInfo.Contains("DOVI")
|| info.Type.Contains("dvvideo")
|| (DoViRegex().IsMatch(output) && info.Type == "Video")
)
info.DolbyVison = true;
if (StartRegex().IsMatch(output))
{
var f = StartRegex().Match(output).Groups[1].Value;
if (double.TryParse(f, out var d))
info.StartTime = TimeSpan.FromSeconds(d);
}
result.Add(info);
}
if (result.Count == 0)
{
result.Add(new Mediainfo
{
Type = "Unknown"
});
}
return result;
}
}

Some files were not shown because too many files have changed in this diff Show More