1
0
Fork 0
forked from loeade/invidious

Compare commits

...

7 commits

Author SHA1 Message Date
23cf582a7a Merge branch 'master' of https://git.xn--la-fka.de/loeade/invidious 2025-04-26 07:26:59 +02:00
Émilien (perso)
0c07e9d27a
chore: set dash by default (#5216) 2025-04-04 14:00:29 +02:00
Émilien (perso)
23ff6135bb
chore: enforce 16 characters for invidious_companion_key (#5220) 2025-03-26 15:27:59 +01:00
syeopite
409d12a81e
Prepare for next release (#5206) 2025-03-16 01:03:01 +00:00
Émilien (perso)
70ff463cc6
Add invidious companion support (#4985)
* add support for invidious companion

* redirect latest_version and dash manifest to invidious companion

* fix Shadowing outer local variable `response`

* fixing condition for Content-Security-Policy

* throw error if inv_sig_helper and invidious_companion used same time

* Use sample instead of Random.rand

Co-authored-by: syeopite <70992037+syeopite@users.noreply.github.com>

* Remove debug puts functions

Co-authored-by: syeopite <70992037+syeopite@users.noreply.github.com>

* modify the description for config.example.yaml about invidious companion

* move config checks for invidious companion

* separate invidious_companion logic + better config.yaml config

* fixing "end" misplacement

* fix linting + use .empty?

* crystal handle decompression already by itself

* fix download function when invidious companion used

* fix linting

* invidious companion always used so always add CSP and redirect latest_version

* apply all the suggestions + rework invidious_companion parameter

* format watch.cr

* fix ameba Redundant use of `Object#to_s` in interpolation

* add ability for invidious companion to check request from invidious

* Better document private_url and public_url

* Better doc for invidious_companion_key

* !empty? to present?

* skip proxy for invidious companion

* fixing format

* missing ,

* add companion pooling http

* fix: don't use http proxy when sending requests to companion

* fix: logic where we want to have the invidious logic if companion is not used

* chore: remove baseurl usage from invidious companion

* chore: change from inv-sig-helper to companion for required playback

* fix: use puts + add warning for inv-sig-helper deprecated

---------

Co-authored-by: syeopite <70992037+syeopite@users.noreply.github.com>
2025-03-13 16:44:00 +01:00
syeopite
e23d0d13be
Add changelog for v2.20250314.0 (#5197)
* Release v2.20250314.0

* Update CHANGELOG.md
2025-03-12 03:31:15 -07:00
syeopite
5c8b4eb379
Warn when po_token, visitor_data and/or inv-sig-helper is not configured (#5202)
* Warn when required configs for playback is missing

* Add link to documentation in warnings

* Direct users to /installation instead
2025-03-12 10:11:17 +01:00
15 changed files with 360 additions and 33 deletions

View file

@ -2,7 +2,102 @@
## vX.Y.0 (future) ## vX.Y.0 (future)
## v2.20250314.0
### Wrap-up
This release brings the long awaited feature of supporting multiple audio tracks in a video, some bug fixes and UX improvements, and many other things primarily oriented to self-hosting instances, and developers using the API.
The `Community` channel tab has been replaced by `Posts` in light of YouTube changes, but the URL remains the same.
Tamil is now available as an interface language
Automatic instance redirects will no longer have the chance to annoyingly redirect to the same instance you're on.
Due to their requirements for video playback, Invidious will log warning messages when either inv-sig-helper, `po_token` or `visitor_data` is not configured
Invidious is now able to listen through a UNIX socket
User notifications are now batched for each channel
**The minimum Crystal version supported by Invidious now `1.12.0`**
### New features & important changes
#### For users
* Invidious now supports videos with multiple audio tracks allowing you to select which one you want to hear with!
* Channel pages now have a proper previous page button
* RSS feeds for channels will no longer contain the channel's profile picture
* Support for channel `courses` page has been added
* `Community` tabs has been replaced with `Posts` to comply with YouTube changes
* Tamil is now an available interface language.
#### For instance owners
* Invidious is now able to listen on a UNIX socket
* User notifications are now batched by channels, significantly reducing database load.
* **`1.12.0` is now the oldest Crystal version that Invidious supports**
* The example config will no longer force an http proxy to be configured
* Invidious will now warn when any top-level config option must be set to a custom value, instead of just `HMAC_KEY`
* Due to their requirements for video playback, Invidious will log warning messages when either inv-sig-helper, `po_token` or `visitor_data` is not configured
#### For developers
* Invidious is now compliant to Crystal 1.15 formatting rules, which are incompatible with earlier versions.
* `/api/v1/transcripts/{id}` has been added to the API to allow for fetching the transcripts for a video. The arguments are the same as the captions endpoint.
* `author_thumbnail` field has been added to videos in the various paged api endpoints
* `published` field has been added to the API response for a video's related videos.
* Docker builds now uses the Crystal compiler cache, reducing build times on repeated builds significantly.
* Invidious ajax action handlers has undergone a clean up and may face compatibility issues with code that depends on these endpoints.
* The versions of Crystal that we test in CI/CD are now: `1.12.1`, `1.13.2`, `1.14.0`, `1.15.0`
### Bugs fixed
#### User-side
* Local video listen mode is now preserved when clicking on a video in the sidebar playlist widget
* Automatic instance redirects will no longer redirect to the same instance the user is on
* Fix some thumbnails responses returning 404
* Videos: Fix missing host parameter on playback URLs when `local=true`
* Fix HLS being used for non-livestream videos
* Fix timeupdate event errors when required elements are missing
* User: Ensure IO is properly closed when importing NewPipe subscriptions
#### For instance owners
* Fix http proxy configuration being forced by the standard example config
#### API
* `/api/v1/videos/{id}` will no longer return an occasional empty JSON response
### Full list of pull requests merged since the last release (newest first)
* Make Invidious compliant to Crystal 1.15 formatting rules (https://github.com/iv-org/invidious/pull/5014, by @syeopite)
* Remove formatter check on container workflows (https://github.com/iv-org/invidious/pull/5153, by @syeopite)
* Videos: Fix missing host parameter on playback URLs when `local=true` (https://github.com/iv-org/invidious/pull/4992, by @SamantazFox)
* Remove stdlib override for proxy initialization (https://github.com/iv-org/invidious/pull/5065, by @syeopite)
* Add support for author thumbnails in search api for videos (https://github.com/iv-org/invidious/pull/5072, thanks @ChunkyProgrammer)
* Skip route if resp got closed by before handlers (https://github.com/iv-org/invidious/pull/5073, by @syeopite)
* Fix video thumbnails in mixes (https://github.com/iv-org/invidious/pull/5116, thanks @iBicha)
* CI: Drop support for versions prior to 1.12 and add 1.15.0 (https://github.com/iv-org/invidious/pull/5148, by @syeopite)
* [Continuing #5094] Set language info for dash audio streams and sort (https://github.com/iv-org/invidious/pull/5149, thanks @giuliano-macedo)
* Warn when any top-level config is "CHANGE_ME!!" (https://github.com/iv-org/invidious/pull/5150, by @syeopite)
* Comment out http_proxy in example config (https://github.com/iv-org/invidious/pull/5151, by @syeopite)
* API: Add a 'published' video parameter for related videos (https://github.com/iv-org/invidious/pull/4149, thanks @RadoslavL)
* Ensure IO is properly closed when importing NewPipe subscriptions (https://github.com/iv-org/invidious/pull/4346, thanks @ChunkyProgrammer)
* Carry over audio-only mode in playlist links (https://github.com/iv-org/invidious/pull/4784, thanks @krystof1119)
* Routes: Clean ajax actions handlers (https://github.com/iv-org/invidious/pull/5036, by @SamantazFox)
* Frontend: Add a first page and previous page buttons for channel navigation (https://github.com/iv-org/invidious/pull/4123, thanks @RadoslavL)
* RSS: Channel + Playlist improvements (https://github.com/iv-org/invidious/pull/4298, thanks @ChunkyProgrammer)
* Batch user notifications together (https://github.com/iv-org/invidious/pull/4486, thanks @999eagle)
* JS: Update timeupdate event making it more defensive to prevent errors (https://github.com/iv-org/invidious/pull/4782, thanks @PMK)
* Add API endpoint for fetching transcripts from YouTube by (https://github.com/iv-org/invidious/pull/4788, by @syeopite)
* Translations update from Hosted Weblate by (https://github.com/iv-org/invidious/pull/4989, thanks to our many translators)
* Add the ability to listen on UNIX sockets (https://github.com/iv-org/invidious/pull/5112, thanks @Caian)
* Pick a different instance upon redirect (https://github.com/iv-org/invidious/pull/5154, thanks @epicsam123)
* Add Courses to channel page and channel API (https://github.com/iv-org/invidious/pull/5158, thanks @ChunkyProgrammer)
* fix /api/v1/videos/:id returns 200 with no content (https://github.com/iv-org/invidious/pull/5162, thanks @Drikanis)
* Use Crystal compiler cache in docker builds (https://github.com/iv-org/invidious/pull/5163, by @syeopite)
* Channels: Fix community tab by (https://github.com/iv-org/invidious/pull/5183, thanks @Fijxu)
* Fix typo in `src/invidious/routes/images.cr` (https://github.com/iv-org/invidious/pull/5184, by @syeopite)
* Fix an issue with the HLS manifest check for livestream videos (https://github.com/iv-org/invidious/pull/5189, thanks @alexmaras)
* Warn when `po_token`, `visitor_data` and/or `inv-sig-helper` is not configured (https://github.com/iv-org/invidious/pull/5202, by @syeopite)
## v2.20241110.0 ## v2.20241110.0
### Wrap-up ### Wrap-up

View file

@ -54,6 +54,53 @@ db:
## ##
#signature_server: #signature_server:
##
## Invidious companion is an external program
## for loading the video streams from YouTube servers.
##
## When this setting is commented out, Invidious companion is not used.
## Otherwise, Invidious will proxy the requests to Invidious companion.
##
## Note: multiple URL can be configured. In this case, invidious will
## randomly pick one every time video data needs to be retrieved. This
## URL is then kept in the video metadata cache to allow video playback
## to work. Once said cache has expired, requesting that video's data
## again will cause a new companion URL to be picked.
##
## The parameter private_url needs to be configured for the internal
## communication between the companion and Invidious.
## And public_url is the public URL from which companion is listening
## to the requests from the user(s).
##
## If you are using a reverse proxy then you will probably need to
## configure the public_url to be the same as the domain used for Invidious.
## Also apply when used from an external IP address (without a domain).
## Examples: https://MYINVIDIOUSDOMAIN or http://192.168.1.100:8282
##
## Both parameter can have identical URL when Invidious is hosted in
## an internal network or at home or locally (localhost).
##
## Accepted values: "http(s)://<IP-HOSTNAME>:<Port>"
## Default: <none>
##
#invidious_companion:
# - private_url: "http://localhost:8282"
# public_url: "http://localhost:8282"
##
## API key for Invidious companion, used for securing the communication
## between Invidious and Invidious companion.
## The size of the key needs to be more or equal to 16.
##
## Note: This parameter is mandatory when Invidious companion is enabled
## and should be a random string.
## Such random string can be generated on linux with the following
## command: `pwgen 16 1`
##
## Accepted values: a string
## Default: <none>
##
#invidious_companion_key: "CHANGE_ME!!"
######################################### #########################################
# #
@ -811,9 +858,9 @@ default_user_preferences:
## Default video quality. ## Default video quality.
## ##
## Accepted values: dash, hd720, medium, small ## Accepted values: dash, hd720, medium, small
## Default: hd720 ## Default: dash
## ##
#quality: hd720 #quality: dash
## ##
## Default dash video quality. ## Default dash video quality.

View file

@ -1,5 +1,5 @@
name: invidious name: invidious
version: 2.20241110.0-dev version: 2.20250314.0-dev
authors: authors:
- Invidious team <contact@invidious.io> - Invidious team <contact@invidious.io>

View file

@ -97,6 +97,10 @@ YT_POOL = YoutubeConnectionPool.new(YT_URL, capacity: CONFIG.pool_size)
GGPHT_POOL = YoutubeConnectionPool.new(URI.parse("https://yt3.ggpht.com"), capacity: CONFIG.pool_size) GGPHT_POOL = YoutubeConnectionPool.new(URI.parse("https://yt3.ggpht.com"), capacity: CONFIG.pool_size)
COMPANION_POOL = CompanionConnectionPool.new(
capacity: CONFIG.pool_size
)
# CLI # CLI
Kemal.config.extra_options do |parser| Kemal.config.extra_options do |parser|
parser.banner = "Usage: invidious [arguments]" parser.banner = "Usage: invidious [arguments]"

View file

@ -35,7 +35,7 @@ struct ConfigPreferences
property max_results : Int32 = 40 property max_results : Int32 = 40
property notifications_only : Bool = false property notifications_only : Bool = false
property player_style : String = "invidious" property player_style : String = "invidious"
property quality : String = "hd720" property quality : String = "dash"
property quality_dash : String = "auto" property quality_dash : String = "auto"
property default_home : String? = "Popular" property default_home : String? = "Popular"
property feed_menu : Array(String) = ["Popular", "Trending", "Subscriptions", "Playlists"] property feed_menu : Array(String) = ["Popular", "Trending", "Subscriptions", "Playlists"]
@ -74,6 +74,16 @@ end
class Config class Config
include YAML::Serializable include YAML::Serializable
class CompanionConfig
include YAML::Serializable
@[YAML::Field(converter: Preferences::URIConverter)]
property private_url : URI = URI.parse("")
@[YAML::Field(converter: Preferences::URIConverter)]
property public_url : URI = URI.parse("")
end
# Number of threads to use for crawling videos from channels (for updating subscriptions) # Number of threads to use for crawling videos from channels (for updating subscriptions)
property channel_threads : Int32 = 1 property channel_threads : Int32 = 1
# Time interval between two executions of the job that crawls channel videos (subscriptions update). # Time interval between two executions of the job that crawls channel videos (subscriptions update).
@ -160,6 +170,12 @@ class Config
# poToken for passing bot attestation # poToken for passing bot attestation
property po_token : String? = nil property po_token : String? = nil
# Invidious companion
property invidious_companion : Array(CompanionConfig) = [] of CompanionConfig
# Invidious companion API key
property invidious_companion_key : String = ""
# Saved cookies in "name1=value1; name2=value2..." format # Saved cookies in "name1=value1; name2=value2..." format
@[YAML::Field(converter: Preferences::StringToCookies)] @[YAML::Field(converter: Preferences::StringToCookies)]
property cookies : HTTP::Cookies = HTTP::Cookies.new property cookies : HTTP::Cookies = HTTP::Cookies.new
@ -240,6 +256,27 @@ class Config
end end
{% end %} {% end %}
if config.invidious_companion.present?
# invidious_companion and signature_server can't work together
if config.signature_server
puts "Config: You can not run inv_sig_helper and invidious_companion at the same time."
exit(1)
elsif config.invidious_companion_key.empty?
puts "Config: Please configure a key if you are using invidious companion."
exit(1)
elsif config.invidious_companion_key == "CHANGE_ME!!"
puts "Config: The value of 'invidious_companion_key' needs to be changed!!"
exit(1)
elsif config.invidious_companion_key.size != 16
puts "Config: The value of 'invidious_companion_key' needs to be a size of 16 characters."
exit(1)
end
elsif config.signature_server
puts("WARNING: inv-sig-helper is deprecated. Please switch to Invidious companion: https://docs.invidious.io/companion-installation/")
else
puts("WARNING: Invidious companion is required to view and playback videos. For more information see https://docs.invidious.io/companion-installation/")
end
# HMAC_key is mandatory # HMAC_key is mandatory
# See: https://github.com/iv-org/invidious/issues/3854 # See: https://github.com/iv-org/invidious/issues/3854
if config.hmac_key.empty? if config.hmac_key.empty?

View file

@ -383,3 +383,22 @@ def parse_link_endpoint(endpoint : JSON::Any, text : String, video_id : String)
end end
return text return text
end end
def encrypt_ecb_without_salt(data, key)
cipher = OpenSSL::Cipher.new("aes-128-ecb")
cipher.encrypt
cipher.key = key
io = IO::Memory.new
io.write(cipher.update(data))
io.write(cipher.final)
io.rewind
return io
end
def invidious_companion_encrypt(data)
timestamp = Time.utc.to_unix
encrypted_data = encrypt_ecb_without_salt("#{timestamp}|#{data}", CONFIG.invidious_companion_key)
return Base64.urlsafe_encode(encrypted_data)
end

View file

@ -8,6 +8,11 @@ module Invidious::Routes::API::Manifest
id = env.params.url["id"] id = env.params.url["id"]
region = env.params.query["region"]? region = env.params.query["region"]?
if CONFIG.invidious_companion.present?
invidious_companion = CONFIG.invidious_companion.sample
return env.redirect "#{invidious_companion.public_url}/api/manifest/dash/id/#{id}?#{env.params.query}"
end
# Since some implementations create playlists based on resolution regardless of different codecs, # Since some implementations create playlists based on resolution regardless of different codecs,
# we can opt to only add a source to a representation if it has a unique height within that representation # we can opt to only add a source to a representation if it has a unique height within that representation
unique_res = env.params.query["unique_res"]?.try { |q| (q == "true" || q == "1").to_unsafe } unique_res = env.params.query["unique_res"]?.try { |q| (q == "true" || q == "1").to_unsafe }

View file

@ -203,6 +203,14 @@ module Invidious::Routes::Embed
return env.redirect url return env.redirect url
end end
if CONFIG.invidious_companion.present?
invidious_companion = CONFIG.invidious_companion.sample
env.response.headers["Content-Security-Policy"] =
env.response.headers["Content-Security-Policy"]
.gsub("media-src", "media-src #{invidious_companion.public_url}")
.gsub("connect-src", "connect-src #{invidious_companion.public_url}")
end
rendered "embed" rendered "embed"
end end
end end

View file

@ -256,6 +256,11 @@ module Invidious::Routes::VideoPlayback
# YouTube /videoplayback links expire after 6 hours, # YouTube /videoplayback links expire after 6 hours,
# so we have a mechanism here to redirect to the latest version # so we have a mechanism here to redirect to the latest version
def self.latest_version(env) def self.latest_version(env)
if CONFIG.invidious_companion.present?
invidious_companion = CONFIG.invidious_companion.sample
return env.redirect "#{invidious_companion.public_url}/latest_version?#{env.params.query}"
end
id = env.params.query["id"]? id = env.params.query["id"]?
itag = env.params.query["itag"]?.try &.to_i? itag = env.params.query["itag"]?.try &.to_i?

View file

@ -192,6 +192,14 @@ module Invidious::Routes::Watch
captions: video.captions captions: video.captions
) )
if CONFIG.invidious_companion.present?
invidious_companion = CONFIG.invidious_companion.sample
env.response.headers["Content-Security-Policy"] =
env.response.headers["Content-Security-Policy"]
.gsub("media-src", "media-src #{invidious_companion.public_url}")
.gsub("connect-src", "connect-src #{invidious_companion.public_url}")
end
templated "watch" templated "watch"
end end
@ -314,14 +322,19 @@ module Invidious::Routes::Watch
env.params.query["label"] = URI.decode_www_form(label.as_s) env.params.query["label"] = URI.decode_www_form(label.as_s)
return Invidious::Routes::API::V1::Videos.captions(env) return Invidious::Routes::API::V1::Videos.captions(env)
elsif itag = download_widget["itag"]?.try &.as_i elsif itag = download_widget["itag"]?.try &.as_i.to_s
# URL params specific to /latest_version # URL params specific to /latest_version
env.params.query["id"] = video_id env.params.query["id"] = video_id
env.params.query["itag"] = itag.to_s
env.params.query["title"] = filename env.params.query["title"] = filename
env.params.query["local"] = "true" env.params.query["local"] = "true"
return Invidious::Routes::VideoPlayback.latest_version(env) if (CONFIG.invidious_companion.present?)
video = get_video(video_id)
invidious_companion = CONFIG.invidious_companion.sample
return env.redirect "#{invidious_companion.public_url}/latest_version?#{env.params.query}"
else
return Invidious::Routes::VideoPlayback.latest_version(env)
end
else else
return error_template(400, "Invalid label or itag") return error_template(400, "Invalid label or itag")
end end

View file

@ -15,7 +15,7 @@ struct Video
# NOTE: don't forget to bump this number if any change is made to # NOTE: don't forget to bump this number if any change is made to
# the `params` structure in videos/parser.cr!!! # the `params` structure in videos/parser.cr!!!
# #
SCHEMA_VERSION = 2 SCHEMA_VERSION = 3
property id : String property id : String

View file

@ -108,27 +108,29 @@ def extract_video_info(video_id : String)
params = parse_video_info(video_id, player_response) params = parse_video_info(video_id, player_response)
params["reason"] = JSON::Any.new(reason) if reason params["reason"] = JSON::Any.new(reason) if reason
new_player_response = nil if !CONFIG.invidious_companion.present?
new_player_response = nil
# Don't use Android test suite client if po_token is passed because po_token doesn't # Don't use Android test suite client if po_token is passed because po_token doesn't
# work for Android test suite client. # work for Android test suite client.
if reason.nil? && CONFIG.po_token.nil? if reason.nil? && CONFIG.po_token.nil?
# Fetch the video streams using an Android client in order to get the # Fetch the video streams using an Android client in order to get the
# decrypted URLs and maybe fix throttling issues (#2194). See the # decrypted URLs and maybe fix throttling issues (#2194). See the
# following issue for an explanation about decrypted URLs: # following issue for an explanation about decrypted URLs:
# https://github.com/TeamNewPipe/NewPipeExtractor/issues/562 # https://github.com/TeamNewPipe/NewPipeExtractor/issues/562
client_config.client_type = YoutubeAPI::ClientType::AndroidTestSuite client_config.client_type = YoutubeAPI::ClientType::AndroidTestSuite
new_player_response = try_fetch_streaming_data(video_id, client_config) new_player_response = try_fetch_streaming_data(video_id, client_config)
end end
# Replace player response and reset reason # Replace player response and reset reason
if !new_player_response.nil? if !new_player_response.nil?
# Preserve captions & storyboard data before replacement # Preserve captions & storyboard data before replacement
new_player_response["storyboards"] = player_response["storyboards"] if player_response["storyboards"]? new_player_response["storyboards"] = player_response["storyboards"] if player_response["storyboards"]?
new_player_response["captions"] = player_response["captions"] if player_response["captions"]? new_player_response["captions"] = player_response["captions"] if player_response["captions"]?
player_response = new_player_response player_response = new_player_response
params.delete("reason") params.delete("reason")
end
end end
{"captions", "playabilityStatus", "playerConfig", "storyboards"}.each do |f| {"captions", "playabilityStatus", "playerConfig", "storyboards"}.each do |f|

View file

@ -22,6 +22,8 @@
audio_streams.each_with_index do |fmt, i| audio_streams.each_with_index do |fmt, i|
src_url = "/latest_version?id=#{video.id}&itag=#{fmt["itag"]}" src_url = "/latest_version?id=#{video.id}&itag=#{fmt["itag"]}"
src_url += "&local=true" if params.local src_url += "&local=true" if params.local
src_url = invidious_companion.public_url.to_s + src_url +
"&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
bitrate = fmt["bitrate"] bitrate = fmt["bitrate"]
mimetype = HTML.escape(fmt["mimeType"].as_s) mimetype = HTML.escape(fmt["mimeType"].as_s)
@ -34,8 +36,12 @@
<% end %> <% end %>
<% end %> <% end %>
<% else %> <% else %>
<% if params.quality == "dash" %> <% if params.quality == "dash"
<source src="/api/manifest/dash/id/<%= video.id %>?local=true&unique_res=1" type='application/dash+xml' label="dash"> src_url = "/api/manifest/dash/id/" + video.id + "?local=true&unique_res=1"
src_url = invidious_companion.public_url.to_s + src_url +
"&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
%>
<source src="<%= src_url %>" type='application/dash+xml' label="dash">
<% end %> <% end %>
<% <%
@ -44,6 +50,8 @@
fmt_stream.each_with_index do |fmt, i| fmt_stream.each_with_index do |fmt, i|
src_url = "/latest_version?id=#{video.id}&itag=#{fmt["itag"]}" src_url = "/latest_version?id=#{video.id}&itag=#{fmt["itag"]}"
src_url += "&local=true" if params.local src_url += "&local=true" if params.local
src_url = invidious_companion.public_url.to_s + src_url +
"&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
quality = fmt["quality"] quality = fmt["quality"]
mimetype = HTML.escape(fmt["mimeType"].as_s) mimetype = HTML.escape(fmt["mimeType"].as_s)

View file

@ -46,6 +46,43 @@ struct YoutubeConnectionPool
end end
end end
struct CompanionConnectionPool
property pool : DB::Pool(HTTP::Client)
def initialize(capacity = 5, timeout = 5.0)
options = DB::Pool::Options.new(
initial_pool_size: 0,
max_pool_size: capacity,
max_idle_pool_size: capacity,
checkout_timeout: timeout
)
@pool = DB::Pool(HTTP::Client).new(options) do
companion = CONFIG.invidious_companion.sample
next make_client(companion.private_url, use_http_proxy: false)
end
end
def client(&)
conn = pool.checkout
begin
response = yield conn
rescue ex
conn.close
companion = CONFIG.invidious_companion.sample
conn = make_client(companion.private_url, use_http_proxy: false)
response = yield conn
ensure
pool.release(conn)
end
response
end
end
def add_yt_headers(request) def add_yt_headers(request)
request.headers.delete("User-Agent") if request.headers["User-Agent"] == "Crystal" request.headers.delete("User-Agent") if request.headers["User-Agent"] == "Crystal"
request.headers["User-Agent"] ||= "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" request.headers["User-Agent"] ||= "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36"
@ -61,9 +98,9 @@ def add_yt_headers(request)
end end
end end
def make_client(url : URI, region = nil, force_resolve : Bool = false, force_youtube_headers : Bool = false) def make_client(url : URI, region = nil, force_resolve : Bool = false, force_youtube_headers : Bool = false, use_http_proxy : Bool = true)
client = HTTP::Client.new(url) client = HTTP::Client.new(url)
client.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy client.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy && use_http_proxy
# Force the usage of a specific configured IP Family # Force the usage of a specific configured IP Family
if force_resolve if force_resolve
@ -78,8 +115,8 @@ def make_client(url : URI, region = nil, force_resolve : Bool = false, force_you
return client return client
end end
def make_client(url : URI, region = nil, force_resolve : Bool = false, &) def make_client(url : URI, region = nil, force_resolve : Bool = false, use_http_proxy : Bool = true, &)
client = make_client(url, region, force_resolve: force_resolve) client = make_client(url, region, force_resolve: force_resolve, use_http_proxy: use_http_proxy)
begin begin
yield client yield client
ensure ensure

View file

@ -500,7 +500,11 @@ module YoutubeAPI
data["params"] = params data["params"] = params
end end
return self._post_json("/youtubei/v1/player", data, client_config) if CONFIG.invidious_companion.present?
return self._post_invidious_companion("/youtubei/v1/player", data)
else
return self._post_json("/youtubei/v1/player", data, client_config)
end
end end
#################################################################### ####################################################################
@ -666,6 +670,49 @@ module YoutubeAPI
return initial_data return initial_data
end end
####################################################################
# _post_invidious_companion(endpoint, data)
#
# Internal function that does the actual request to Invidious companion
# and handles errors.
#
# The requested data is an endpoint (URL without the domain part)
# and the data as a Hash object.
#
def _post_invidious_companion(
endpoint : String,
data : Hash,
) : Hash(String, JSON::Any)
headers = HTTP::Headers{
"Content-Type" => "application/json; charset=UTF-8",
"Authorization" => "Bearer #{CONFIG.invidious_companion_key}",
}
# Logging
LOGGER.debug("Invidious companion: Using endpoint: \"#{endpoint}\"")
LOGGER.trace("Invidious companion: POST data: #{data}")
# Send the POST request
begin
response = COMPANION_POOL.client &.post(endpoint, headers: headers, body: data.to_json)
body = response.body
if (response.status_code != 200)
raise Exception.new(
"Error while communicating with Invidious companion: \
status code: #{response.status_code} and body: #{body.dump}"
)
end
rescue ex
raise InfoException.new("Error while communicating with Invidious companion: " + (ex.message || "no extra info found"))
end
# Convert result to Hash
initial_data = JSON.parse(body).as_h
return initial_data
end
#################################################################### ####################################################################
# _decompress(body_io, headers) # _decompress(body_io, headers)
# #