1
0
Fork 0
forked from loeade/invidious

Compare commits

..

No commits in common. "55bf2e719b92e51f129e76606a99ad82f0e62079" and "2fa96fc31a22b0ec76d64e0d76b9f7f865453e87" have entirely different histories.

48 changed files with 282 additions and 308 deletions

View file

@ -10,10 +10,8 @@ assignees: ''
<!-- <!--
BEFORE TRYING TO REPORT A BUG: BEFORE TRYING TO REPORT A BUG:
* Read the FAQ: https://docs.invidious.io/faq/! * Read the FAQ!
* Use the search function to check if there is already an issue open for your problem: https://github.com/search?q=repo%3Aiv-org%2Finvidious+replace+me+with+your+bug&type=issues! * Use the search function to check if there is already an issue open for your problem!
MAKE SURE TO FOLLOW THE TWO STEPS ABOVE BEFORE REPORTING A BUG. A BUG THAT ALREADY EXIST WILL IMMEDIATELY CLOSED.
If you want to suggest a new feature please use "Feature request" instead If you want to suggest a new feature please use "Feature request" instead
If you want to suggest an enhancement to an existing feature please use "Enhancement" instead If you want to suggest an enhancement to an existing feature please use "Enhancement" instead

View file

@ -23,6 +23,19 @@ jobs:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Install Crystal
uses: crystal-lang/install-crystal@v1.8.2
with:
crystal: 1.12.2
- name: Run lint
run: |
if ! crystal tool format --check; then
crystal tool format
git diff
exit 1
fi
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v3 uses: docker/setup-qemu-action@v3
with: with:

View file

@ -14,6 +14,19 @@ jobs:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Install Crystal
uses: crystal-lang/install-crystal@v1.8.2
with:
crystal: 1.12.2
- name: Run lint
run: |
if ! crystal tool format --check; then
crystal tool format
git diff
exit 1
fi
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v3 uses: docker/setup-qemu-action@v3
with: with:

View file

@ -38,10 +38,11 @@ jobs:
matrix: matrix:
stable: [true] stable: [true]
crystal: crystal:
- 1.10.1
- 1.11.2
- 1.12.1 - 1.12.1
- 1.13.2 - 1.13.2
- 1.14.0 - 1.14.0
- 1.15.0
include: include:
- crystal: nightly - crystal: nightly
stable: false stable: false
@ -135,7 +136,6 @@ jobs:
submodules: true submodules: true
- name: Install Crystal - name: Install Crystal
id: lint_step_install_crystal
uses: crystal-lang/install-crystal@v1.8.0 uses: crystal-lang/install-crystal@v1.8.0
with: with:
crystal: latest crystal: latest
@ -146,7 +146,7 @@ jobs:
path: | path: |
./lib ./lib
./bin ./bin
key: shards-${{ hashFiles('shard.lock') }}-${{ steps.lint_step_install_crystal.outputs.crystal }} key: shards-${{ hashFiles('shard.lock') }}
- name: Install Shards - name: Install Shards
run: | run: |

View file

@ -3,84 +3,8 @@
## vX.Y.0 (future) ## vX.Y.0 (future)
## v2.20241110.0
### Wrap-up
This release is most importantly here to fix to the annoying "Youtube API returned error 400"
error that prevented all channel pages from loading.
If you're updating from the previous release, it provides no improvements on the ability to play
videos. If updating from a commit in-between release, it removes the "Please sign in" error caused
by a previous attempt at restoring video playback on large instances.
In the preferences, a new option allows for control of video preload. When enabled, this option
tells the browser to load the video as soon as the page is loaded (this used to be the default).
When disabled, the video starts loading only when the "play" button is pressed.
New interface languages available: Bulgarian, Welsh and Lombard
New dependency required: `tzdata`.
An HTTP proxy can be configured directly in Invidious, if needed. \
**NOTE:** In that case, it is recommended to comment out `force_resolve`.
### New features & important changes
#### For users
* Channels: Fix "Youtube API returned error 400" error preventing channel pages from loading
* Channels: Shorts can now be sorted by "newest", "oldest" and "popular"
* Preferences: Addition of the new "preload" option
* New interface languages available: Bulgarian, Welsh and Lombard
* Added "Filipino (auto-generated)" to the list of caption languages available
* Lots of new translations from Weblate
#### For instance owners
* Allow the configuration of an HTTP proxy to talk to Youtube
* Invidious tries to reconnect to `inv_sig_helper` if the socket is closed
* The instance list is downloaded in the background to improve redirection speed
* New `colorize_logs` option makes each log level a different color
#### For developpers
* `/api/v1/channels/{id}/shorts` now supports the `sort-by` parameter with the following values:
`newest`, `oldest` and `popular`
* Older `/api/v1/channels/xyz/{id}` (tab name before UCID) were removed
* API/Search: New video metadata available: `isNew`, `is4k`, `is8k`, `isVr180`, `isVr360`,
`is3d` and `hasCaptions`
### Bugs fixed
#### User-side
* Channels: The second page of shorts now loads as expected
* Channels: Fixed intermittent empty "playlists" tab
* Search: Fixed `youtu.be` URLs not being properly redirected to the watch page
* Fixed `DB::MappingException` error on the subscriptions feed (due to missing `tzdata` in docker)
* Switching to another instance is much faster
* Fixed an "invalid byte sequence" error when subscribing to a playlist
* Videos: Playback URLs were sometimes broken when cached and `inv_sig_helper` was used
#### For instance owners
* Fix `force_resolve` being ignored in some cases
#### API
* API/Videos: Fixed `live_now` and `premiere_timestamp` sometimes not having the right values
### Full list of pull requests merged since the last release (newest first) ### Full list of pull requests merged since the last release (newest first)
* API: Add "sort_by" parameter to channels/shorts endpoint ([#5071], thanks @iBicha)
* Docker: Install tzdata in Dockerfile ([#5070], by @SamantazFox)
* Videos: Stop using TVHTML5_SIMPLY_EMBEDDED_PLAYER ([#5063], thanks @unixfox)
* Routing: Deprecate old channel API routes ([#5045], by @SamantazFox)
* Videos: use WEB client instead of WEB CREATOR ([#4984], thanks @unixfox)
* Parsers: Fix parsing live_now and premiere_timestamp ([#4934], thanks @absidue)
* Stale bot updates ([#5060], thanks @syeopite) * Stale bot updates ([#5060], thanks @syeopite)
* Channels: Fix "Youtube API returned error 400" ([#5059], by @SamantazFox) * Channels: Fix "Youtube API returned error 400" ([#5059], by @SamantazFox)
* Channels: Fix for live videos ([#5027], thanks @iBicha) * Channels: Fix for live videos ([#5027], thanks @iBicha)
@ -128,21 +52,15 @@ An HTTP proxy can be configured directly in Invidious, if needed. \
[#4928]: https://github.com/iv-org/invidious/pull/4928 [#4928]: https://github.com/iv-org/invidious/pull/4928
[#4930]: https://github.com/iv-org/invidious/pull/4930 [#4930]: https://github.com/iv-org/invidious/pull/4930
[#4931]: https://github.com/iv-org/invidious/pull/4931 [#4931]: https://github.com/iv-org/invidious/pull/4931
[#4934]: https://github.com/iv-org/invidious/pull/4934
[#4942]: https://github.com/iv-org/invidious/pull/4942 [#4942]: https://github.com/iv-org/invidious/pull/4942
[#4984]: https://github.com/iv-org/invidious/pull/4984
[#4991]: https://github.com/iv-org/invidious/pull/4991 [#4991]: https://github.com/iv-org/invidious/pull/4991
[#4993]: https://github.com/iv-org/invidious/pull/4993 [#4993]: https://github.com/iv-org/invidious/pull/4993
[#4995]: https://github.com/iv-org/invidious/pull/4995 [#4995]: https://github.com/iv-org/invidious/pull/4995
[#5027]: https://github.com/iv-org/invidious/pull/5027 [#5027]: https://github.com/iv-org/invidious/pull/5027
[#5034]: https://github.com/iv-org/invidious/pull/5034 [#5034]: https://github.com/iv-org/invidious/pull/5034
[#5045]: https://github.com/iv-org/invidious/pull/5045
[#5046]: https://github.com/iv-org/invidious/pull/5046 [#5046]: https://github.com/iv-org/invidious/pull/5046
[#5059]: https://github.com/iv-org/invidious/pull/5059 [#5059]: https://github.com/iv-org/invidious/pull/5059
[#5060]: https://github.com/iv-org/invidious/pull/5060 [#5060]: https://github.com/iv-org/invidious/pull/5060
[#5063]: https://github.com/iv-org/invidious/pull/5063
[#5070]: https://github.com/iv-org/invidious/pull/5070
[#5071]: https://github.com/iv-org/invidious/pull/5071
## v2.20240825.2 (2024-08-26) ## v2.20240825.2 (2024-08-26)

View file

@ -91,7 +91,7 @@
var count = document.getElementById('count'); var count = document.getElementById('count');
count.textContent--; count.textContent--;
var url = '/token_ajax?action=revoke_token&redirect=false' + var url = '/token_ajax?action_revoke_token=1&redirect=false' +
'&referer=' + encodeURIComponent(location.href) + '&referer=' + encodeURIComponent(location.href) +
'&session=' + target.getAttribute('data-session'); '&session=' + target.getAttribute('data-session');
@ -111,7 +111,7 @@
var count = document.getElementById('count'); var count = document.getElementById('count');
count.textContent--; count.textContent--;
var url = '/subscription_ajax?action=remove_subscriptions&redirect=false' + var url = '/subscription_ajax?action_remove_subscriptions=1&redirect=false' +
'&referer=' + encodeURIComponent(location.href) + '&referer=' + encodeURIComponent(location.href) +
'&c=' + target.getAttribute('data-ucid'); '&c=' + target.getAttribute('data-ucid');

View file

@ -6,7 +6,7 @@ function add_playlist_video(target) {
var select = target.parentNode.children[0].children[1]; var select = target.parentNode.children[0].children[1];
var option = select.children[select.selectedIndex]; var option = select.children[select.selectedIndex];
var url = '/playlist_ajax?action=add_video&redirect=false' + var url = '/playlist_ajax?action_add_video=1&redirect=false' +
'&video_id=' + target.getAttribute('data-id') + '&video_id=' + target.getAttribute('data-id') +
'&playlist_id=' + option.getAttribute('data-plid'); '&playlist_id=' + option.getAttribute('data-plid');
@ -21,7 +21,7 @@ function add_playlist_item(target) {
var tile = target.parentNode.parentNode.parentNode.parentNode.parentNode; var tile = target.parentNode.parentNode.parentNode.parentNode.parentNode;
tile.style.display = 'none'; tile.style.display = 'none';
var url = '/playlist_ajax?action=add_video&redirect=false' + var url = '/playlist_ajax?action_add_video=1&redirect=false' +
'&video_id=' + target.getAttribute('data-id') + '&video_id=' + target.getAttribute('data-id') +
'&playlist_id=' + target.getAttribute('data-plid'); '&playlist_id=' + target.getAttribute('data-plid');
@ -36,7 +36,7 @@ function remove_playlist_item(target) {
var tile = target.parentNode.parentNode.parentNode.parentNode.parentNode; var tile = target.parentNode.parentNode.parentNode.parentNode.parentNode;
tile.style.display = 'none'; tile.style.display = 'none';
var url = '/playlist_ajax?action=remove_video&redirect=false' + var url = '/playlist_ajax?action_remove_video=1&redirect=false' +
'&set_video_id=' + target.getAttribute('data-index') + '&set_video_id=' + target.getAttribute('data-index') +
'&playlist_id=' + target.getAttribute('data-plid'); '&playlist_id=' + target.getAttribute('data-plid');

View file

@ -16,7 +16,7 @@ function subscribe() {
subscribe_button.onclick = unsubscribe; subscribe_button.onclick = unsubscribe;
subscribe_button.innerHTML = '<b>' + subscribe_data.unsubscribe_text + ' | ' + subscribe_data.sub_count_text + '</b>'; subscribe_button.innerHTML = '<b>' + subscribe_data.unsubscribe_text + ' | ' + subscribe_data.sub_count_text + '</b>';
var url = '/subscription_ajax?action=create_subscription_to_channel&redirect=false' + var url = '/subscription_ajax?action_create_subscription_to_channel=1&redirect=false' +
'&c=' + subscribe_data.ucid; '&c=' + subscribe_data.ucid;
helpers.xhr('POST', url, {payload: payload, retries: 5, entity_name: 'subscribe request'}, { helpers.xhr('POST', url, {payload: payload, retries: 5, entity_name: 'subscribe request'}, {
@ -32,7 +32,7 @@ function unsubscribe() {
subscribe_button.onclick = subscribe; subscribe_button.onclick = subscribe;
subscribe_button.innerHTML = '<b>' + subscribe_data.subscribe_text + ' | ' + subscribe_data.sub_count_text + '</b>'; subscribe_button.innerHTML = '<b>' + subscribe_data.subscribe_text + ' | ' + subscribe_data.sub_count_text + '</b>';
var url = '/subscription_ajax?action=remove_subscriptions&redirect=false' + var url = '/subscription_ajax?action_remove_subscriptions=1&redirect=false' +
'&c=' + subscribe_data.ucid; '&c=' + subscribe_data.ucid;
helpers.xhr('POST', url, {payload: payload, retries: 5, entity_name: 'unsubscribe request'}, { helpers.xhr('POST', url, {payload: payload, retries: 5, entity_name: 'unsubscribe request'}, {

View file

@ -67,10 +67,6 @@ function get_playlist(plid) {
'&format=html&hl=' + video_data.preferences.locale; '&format=html&hl=' + video_data.preferences.locale;
} }
if (video_data.params.listen) {
plid_url += '&listen=1'
}
helpers.xhr('GET', plid_url, {retries: 5, entity_name: 'playlist'}, { helpers.xhr('GET', plid_url, {retries: 5, entity_name: 'playlist'}, {
on200: function (response) { on200: function (response) {
playlist.innerHTML = response.playlistHtml; playlist.innerHTML = response.playlistHtml;

View file

@ -6,7 +6,7 @@ function mark_watched(target) {
var tile = target.parentNode.parentNode.parentNode.parentNode.parentNode; var tile = target.parentNode.parentNode.parentNode.parentNode.parentNode;
tile.style.display = 'none'; tile.style.display = 'none';
var url = '/watch_ajax?action=mark_watched&redirect=false' + var url = '/watch_ajax?action_mark_watched=1&redirect=false' +
'&id=' + target.getAttribute('data-id'); '&id=' + target.getAttribute('data-id');
helpers.xhr('POST', url, {payload: payload}, { helpers.xhr('POST', url, {payload: payload}, {
@ -22,7 +22,7 @@ function mark_unwatched(target) {
var count = document.getElementById('count'); var count = document.getElementById('count');
count.textContent--; count.textContent--;
var url = '/watch_ajax?action=mark_unwatched&redirect=false' + var url = '/watch_ajax?action_mark_unwatched=1&redirect=false' +
'&id=' + target.getAttribute('data-id'); '&id=' + target.getAttribute('data-id');
helpers.xhr('POST', url, {payload: payload}, { helpers.xhr('POST', url, {payload: payload}, {

View file

@ -178,11 +178,11 @@ https_only: false
## ##
## If unset, then no HTTP proxy will be used. ## If unset, then no HTTP proxy will be used.
## ##
#http_proxy: http_proxy:
# user: user:
# password: password:
# host: host:
# port: port:
## ##

View file

@ -1,4 +1,4 @@
FROM crystallang/crystal:1.12.2-alpine AS builder FROM crystallang/crystal:1.12.1-alpine AS builder
RUN apk add --no-cache sqlite-static yaml-static RUN apk add --no-cache sqlite-static yaml-static
@ -32,8 +32,8 @@ RUN if [[ "${release}" == 1 ]] ; then \
--link-flags "-lxml2 -llzma"; \ --link-flags "-lxml2 -llzma"; \
fi fi
FROM alpine:3.20 FROM alpine:3.18
RUN apk add --no-cache rsvg-convert ttf-opensans tini tzdata RUN apk add --no-cache rsvg-convert ttf-opensans tini
WORKDIR /invidious WORKDIR /invidious
RUN addgroup -g 1000 -S invidious && \ RUN addgroup -g 1000 -S invidious && \
adduser -u 1000 -S invidious -G invidious adduser -u 1000 -S invidious -G invidious

View file

@ -1,6 +1,5 @@
FROM alpine:3.20 AS builder FROM alpine:3.19 AS builder
RUN apk add --no-cache 'crystal=1.12.2-r0' shards sqlite-static yaml-static yaml-dev libxml2-static \ RUN apk add --no-cache 'crystal=1.10.1-r0' shards sqlite-static yaml-static yaml-dev libxml2-static zlib-static openssl-libs-static openssl-dev musl-dev xz-static
zlib-static openssl-libs-static openssl-dev musl-dev xz-static
ARG release ARG release
@ -33,8 +32,8 @@ RUN if [[ "${release}" == 1 ]] ; then \
--link-flags "-lxml2 -llzma"; \ --link-flags "-lxml2 -llzma"; \
fi fi
FROM alpine:3.20 FROM alpine:3.18
RUN apk add --no-cache rsvg-convert ttf-opensans tini tzdata RUN apk add --no-cache rsvg-convert ttf-opensans tini
WORKDIR /invidious WORKDIR /invidious
RUN addgroup -g 1000 -S invidious && \ RUN addgroup -g 1000 -S invidious && \
adduser -u 1000 -S invidious -G invidious adduser -u 1000 -S invidious -G invidious

View file

@ -1,12 +1,13 @@
name: invidious name: invidious
version: 2.20241110.0-dev version: 0.20.1
authors: authors:
- Invidious team <contact@invidious.io> - Omar Roth <omarroth@protonmail.com>
- Contributors! - Invidious team
description: | targets:
Invidious is an alternative front-end to YouTube invidious:
main: src/invidious.cr
dependencies: dependencies:
pg: pg:
@ -39,10 +40,6 @@ development_dependencies:
github: crystal-ameba/ameba github: crystal-ameba/ameba
version: ~> 1.6.1 version: ~> 1.6.1
crystal: ">= 1.10.0, < 2.0.0" crystal: ">= 1.0.0, < 2.0.0"
license: AGPLv3 license: AGPLv3
repository: https://github.com/iv-org/invidious
homepage: https://invidious.io
documentation: https://docs.invidious.io

View file

@ -184,9 +184,6 @@ class Config
config = Config.from_yaml(config_yaml) config = Config.from_yaml(config_yaml)
# Update config from env vars (upcased and prefixed with "INVIDIOUS_") # Update config from env vars (upcased and prefixed with "INVIDIOUS_")
#
# Also checks if any top-level config options are set to "CHANGE_ME!!"
# TODO: Support non-top-level config options such as the ones in DBConfig
{% for ivar in Config.instance_vars %} {% for ivar in Config.instance_vars %}
{% env_id = "INVIDIOUS_#{ivar.id.upcase}" %} {% env_id = "INVIDIOUS_#{ivar.id.upcase}" %}
@ -223,12 +220,6 @@ class Config
exit(1) exit(1)
end end
end end
# Warn when any config attribute is set to "CHANGE_ME!!"
if config.{{ivar.id}} == "CHANGE_ME!!"
puts "Config: The value of '#{ {{ivar.stringify}} }' needs to be changed!!"
exit(1)
end
{% end %} {% end %}
# HMAC_key is mandatory # HMAC_key is mandatory
@ -236,6 +227,9 @@ class Config
if config.hmac_key.empty? if config.hmac_key.empty?
puts "Config: 'hmac_key' is required/can't be empty" puts "Config: 'hmac_key' is required/can't be empty"
exit(1) exit(1)
elsif config.hmac_key == "CHANGE_ME!!"
puts "Config: The value of 'hmac_key' needs to be changed!!"
exit(1)
end end
# Build database_url from db.* if it's not set directly # Build database_url from db.* if it's not set directly

View file

@ -13,7 +13,7 @@ module Invidious::Frontend::WatchPage
@full_videos, @full_videos,
@video_streams, @video_streams,
@audio_streams, @audio_streams,
@captions, @captions
) )
end end
end end

View file

@ -18,6 +18,40 @@ end
class HTTP::Client class HTTP::Client
property family : Socket::Family = Socket::Family::UNSPEC property family : Socket::Family = Socket::Family::UNSPEC
# Override stdlib to automatically initialize proxy if configured
#
# Accurate as of crystal 1.12.1
def initialize(@host : String, port = nil, tls : TLSContext = nil)
check_host_only(@host)
{% if flag?(:without_openssl) %}
if tls
raise "HTTP::Client TLS is disabled because `-D without_openssl` was passed at compile time"
end
@tls = nil
{% else %}
@tls = case tls
when true
OpenSSL::SSL::Context::Client.new
when OpenSSL::SSL::Context::Client
tls
when false, nil
nil
end
{% end %}
@port = (port || (@tls ? 443 : 80)).to_i
self.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy
end
def initialize(@io : IO, @host = "", @port = 80)
@reconnect = false
self.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy
end
private def io private def io
io = @io io = @io
return io if io return io if io

View file

@ -130,7 +130,7 @@ def error_json_helper(
env : HTTP::Server::Context, env : HTTP::Server::Context,
status_code : Int32, status_code : Int32,
exception : Exception, exception : Exception,
additional_fields : Hash(String, Object) | Nil = nil, additional_fields : Hash(String, Object) | Nil = nil
) )
if exception.is_a?(InfoException) if exception.is_a?(InfoException)
return error_json_helper(env, status_code, exception.message || "", additional_fields) return error_json_helper(env, status_code, exception.message || "", additional_fields)
@ -152,7 +152,7 @@ def error_json_helper(
env : HTTP::Server::Context, env : HTTP::Server::Context,
status_code : Int32, status_code : Int32,
message : String, message : String,
additional_fields : Hash(String, Object) | Nil = nil, additional_fields : Hash(String, Object) | Nil = nil
) )
env.response.content_type = "application/json" env.response.content_type = "application/json"
env.response.status_code = status_code env.response.status_code = status_code

View file

@ -27,7 +27,6 @@ class Kemal::RouteHandler
# Processes the route if it's a match. Otherwise renders 404. # Processes the route if it's a match. Otherwise renders 404.
private def process_request(context) private def process_request(context)
raise Kemal::Exceptions::RouteNotFound.new(context) unless context.route_found? raise Kemal::Exceptions::RouteNotFound.new(context) unless context.route_found?
return if context.response.closed?
content = context.route.handler.call(context) content = context.route.handler.call(context)
if !Kemal.config.error_handlers.empty? && Kemal.config.error_handlers.has_key?(context.response.status_code) && exclude_match?(context) if !Kemal.config.error_handlers.empty? && Kemal.config.error_handlers.has_key?(context.response.status_code) && exclude_match?(context)

View file

@ -24,7 +24,6 @@ struct SearchVideo
property length_seconds : Int32 property length_seconds : Int32
property premiere_timestamp : Time? property premiere_timestamp : Time?
property author_verified : Bool property author_verified : Bool
property author_thumbnail : String?
property badges : VideoBadges property badges : VideoBadges
def to_xml(auto_generated, query_params, xml : XML::Builder) def to_xml(auto_generated, query_params, xml : XML::Builder)
@ -89,24 +88,6 @@ struct SearchVideo
json.field "authorUrl", "/channel/#{self.ucid}" json.field "authorUrl", "/channel/#{self.ucid}"
json.field "authorVerified", self.author_verified json.field "authorVerified", self.author_verified
author_thumbnail = self.author_thumbnail
if author_thumbnail
json.field "authorThumbnails" do
json.array do
qualities = {32, 48, 76, 100, 176, 512}
qualities.each do |quality|
json.object do
json.field "url", author_thumbnail.gsub(/=s\d+/, "=s#{quality}")
json.field "width", quality
json.field "height", quality
end
end
end
end
end
json.field "videoThumbnails" do json.field "videoThumbnails" do
Invidious::JSONify::APIv1.thumbnails(json, self.id) Invidious::JSONify::APIv1.thumbnails(json, self.id)
end end
@ -242,7 +223,7 @@ struct SearchChannel
qualities.each do |quality| qualities.each do |quality|
json.object do json.object do
json.field "url", self.author_thumbnail.gsub(/=s\d+/, "=s#{quality}") json.field "url", self.author_thumbnail.gsub(/=\d+/, "=s#{quality}")
json.field "width", quality json.field "width", quality
json.field "height", quality json.field "height", quality
end end

View file

@ -267,12 +267,6 @@ module Invidious::JSONify::APIv1
json.field "lengthSeconds", rv["length_seconds"]?.try &.to_i json.field "lengthSeconds", rv["length_seconds"]?.try &.to_i
json.field "viewCountText", rv["short_view_count"]? json.field "viewCountText", rv["short_view_count"]?
json.field "viewCount", rv["view_count"]?.try &.empty? ? nil : rv["view_count"].to_i64 json.field "viewCount", rv["view_count"]?.try &.empty? ? nil : rv["view_count"].to_i64
json.field "published", rv["published"]?
if !rv["published"]?.nil?
json.field "publishedText", translate(locale, "`x` ago", recode_date(Time.parse_rfc3339(rv["published"].to_s), locale))
else
json.field "publishedText", ""
end
end end
end end
end end

View file

@ -81,7 +81,7 @@ def fetch_mix(rdid, video_id, cookies = nil, locale = nil)
}) })
end end
def template_mix(mix, listen) def template_mix(mix)
html = <<-END_HTML html = <<-END_HTML
<h3> <h3>
<a href="/mix?list=#{mix["mixId"]}"> <a href="/mix?list=#{mix["mixId"]}">
@ -95,7 +95,7 @@ def template_mix(mix, listen)
mix["videos"].as_a.each do |video| mix["videos"].as_a.each do |video|
html += <<-END_HTML html += <<-END_HTML
<li class="pure-menu-item"> <li class="pure-menu-item">
<a href="/watch?v=#{video["videoId"]}&list=#{mix["mixId"]}#{listen ? "&listen=1" : ""}"> <a href="/watch?v=#{video["videoId"]}&list=#{mix["mixId"]}">
<div class="thumbnail"> <div class="thumbnail">
<img loading="lazy" class="thumbnail" src="/vi/#{video["videoId"]}/mqdefault.jpg" alt="" /> <img loading="lazy" class="thumbnail" src="/vi/#{video["videoId"]}/mqdefault.jpg" alt="" />
<p class="length">#{recode_length_seconds(video["lengthSeconds"].as_i)}</p> <p class="length">#{recode_length_seconds(video["lengthSeconds"].as_i)}</p>

View file

@ -505,7 +505,7 @@ def extract_playlist_videos(initial_data : Hash(String, JSON::Any))
return videos return videos
end end
def template_playlist(playlist, listen) def template_playlist(playlist)
html = <<-END_HTML html = <<-END_HTML
<h3> <h3>
<a href="/playlist?list=#{playlist["playlistId"]}"> <a href="/playlist?list=#{playlist["playlistId"]}">
@ -519,7 +519,7 @@ def template_playlist(playlist, listen)
playlist["videos"].as_a.each do |video| playlist["videos"].as_a.each do |video|
html += <<-END_HTML html += <<-END_HTML
<li class="pure-menu-item" id="#{video["videoId"]}"> <li class="pure-menu-item" id="#{video["videoId"]}">
<a href="/watch?v=#{video["videoId"]}&list=#{playlist["playlistId"]}&index=#{video["index"]}#{listen ? "&listen=1" : ""}"> <a href="/watch?v=#{video["videoId"]}&list=#{playlist["playlistId"]}&index=#{video["index"]}">
<div class="thumbnail"> <div class="thumbnail">
<img loading="lazy" class="thumbnail" src="/vi/#{video["videoId"]}/mqdefault.jpg" alt="" /> <img loading="lazy" class="thumbnail" src="/vi/#{video["videoId"]}/mqdefault.jpg" alt="" />
<p class="length">#{recode_length_seconds(video["lengthSeconds"].as_i)}</p> <p class="length">#{recode_length_seconds(video["lengthSeconds"].as_i)}</p>

View file

@ -328,9 +328,17 @@ module Invidious::Routes::Account
end end
end end
case action = env.params.query["action"]? if env.params.query["action_revoke_token"]?
when "revoke_token" action = "action_revoke_token"
session = env.params.query["session"] else
return env.redirect referer
end
session = env.params.query["session"]?
session ||= ""
case action
when .starts_with? "action_revoke_token"
Invidious::Database::SessionIDs.delete(sid: session, email: user.email) Invidious::Database::SessionIDs.delete(sid: session, email: user.email)
else else
return error_json(400, "Unsupported action #{action}") return error_json(400, "Unsupported action #{action}")

View file

@ -27,21 +27,28 @@ module Invidious::Routes::API::Manifest
haltf env, status_code: response.status_code haltf env, status_code: response.status_code
end end
# Proxy URLs for video playback on invidious. manifest = response.body
# Other API clients can get the original URLs by omiting `local=true`.
manifest = response.body.gsub(/<BaseURL>[^<]+<\/BaseURL>/) do |baseurl| manifest = manifest.gsub(/<BaseURL>[^<]+<\/BaseURL>/) do |baseurl|
url = baseurl.lchop("<BaseURL>").rchop("</BaseURL>") url = baseurl.lchop("<BaseURL>")
url = HttpServer::Utils.proxy_video_url(url, absolute: true) if local url = url.rchop("</BaseURL>")
if local
uri = URI.parse(url)
url = "#{HOST_URL}#{uri.request_target}host/#{uri.host}/"
end
"<BaseURL>#{url}</BaseURL>" "<BaseURL>#{url}</BaseURL>"
end end
return manifest return manifest
end end
# Ditto, only proxify URLs if `local=true` is used adaptive_fmts = video.adaptive_fmts
if local if local
video.adaptive_fmts.each do |fmt| adaptive_fmts.each do |fmt|
fmt["url"] = JSON::Any.new(HttpServer::Utils.proxy_video_url(fmt["url"].as_s, absolute: true)) fmt["url"] = JSON::Any.new("#{HOST_URL}#{URI.parse(fmt["url"].as_s).request_target}")
end end
end end
@ -63,23 +70,17 @@ module Invidious::Routes::API::Manifest
# OTF streams aren't supported yet (See https://github.com/TeamNewPipe/NewPipe/issues/2415) # OTF streams aren't supported yet (See https://github.com/TeamNewPipe/NewPipe/issues/2415)
next if !(fmt.has_key?("indexRange") && fmt.has_key?("initRange")) next if !(fmt.has_key?("indexRange") && fmt.has_key?("initRange"))
audio_track = fmt["audioTrack"]?.try &.as_h? || {} of String => JSON::Any
lang = audio_track["id"]?.try &.as_s.split('.')[0] || "und"
is_default = audio_track.has_key?("audioIsDefault") ? audio_track["audioIsDefault"].as_bool : i == 0
displayname = audio_track["displayName"]?.try &.as_s || "Unknown"
bitrate = fmt["bitrate"]
# Different representations of the same audio should be groupped into one AdaptationSet. # Different representations of the same audio should be groupped into one AdaptationSet.
# However, most players don't support auto quality switching, so we have to trick them # However, most players don't support auto quality switching, so we have to trick them
# into providing a quality selector. # into providing a quality selector.
# See https://github.com/iv-org/invidious/issues/3074 for more details. # See https://github.com/iv-org/invidious/issues/3074 for more details.
xml.element("AdaptationSet", id: i, mimeType: mime_type, startWithSAP: 1, subsegmentAlignment: true, label: "#{displayname} [#{bitrate}k]", lang: lang) do xml.element("AdaptationSet", id: i, mimeType: mime_type, startWithSAP: 1, subsegmentAlignment: true, label: fmt["bitrate"].to_s + "k") do
codecs = fmt["mimeType"].as_s.split("codecs=")[1].strip('"') codecs = fmt["mimeType"].as_s.split("codecs=")[1].strip('"')
bandwidth = fmt["bitrate"].as_i bandwidth = fmt["bitrate"].as_i
itag = fmt["itag"].as_i itag = fmt["itag"].as_i
url = fmt["url"].as_s url = fmt["url"].as_s
xml.element("Role", schemeIdUri: "urn:mpeg:dash:role:2011", value: is_default ? "main" : "alternate") xml.element("Role", schemeIdUri: "urn:mpeg:dash:role:2011", value: i == 0 ? "main" : "alternate")
xml.element("Representation", id: fmt["itag"], codecs: codecs, bandwidth: bandwidth) do xml.element("Representation", id: fmt["itag"], codecs: codecs, bandwidth: bandwidth) do
xml.element("AudioChannelConfiguration", schemeIdUri: "urn:mpeg:dash:23003:3:audio_channel_configuration:2011", xml.element("AudioChannelConfiguration", schemeIdUri: "urn:mpeg:dash:23003:3:audio_channel_configuration:2011",
@ -176,9 +177,8 @@ module Invidious::Routes::API::Manifest
manifest = response.body manifest = response.body
if local if local
manifest = manifest.gsub(/https:\/\/[^\n"]*/m) do |match| manifest = manifest.gsub(/^https:\/\/\w+---.{11}\.c\.youtube\.com[^\n]*/m) do |match|
uri = URI.parse(match) path = URI.parse(match).path
path = uri.path
path = path.lchop("/videoplayback/") path = path.lchop("/videoplayback/")
path = path.rchop("/") path = path.rchop("/")
@ -207,7 +207,7 @@ module Invidious::Routes::API::Manifest
raw_params["fvip"] = fvip["fvip"] raw_params["fvip"] = fvip["fvip"]
end end
raw_params["host"] = uri.host.not_nil! raw_params["local"] = "true"
"#{HOST_URL}/videoplayback?#{raw_params}" "#{HOST_URL}/videoplayback?#{raw_params}"
end end

View file

@ -197,7 +197,6 @@ module Invidious::Routes::API::V1::Channels
get_channel() get_channel()
# Retrieve continuation from URL parameters # Retrieve continuation from URL parameters
sort_by = env.params.query["sort_by"]?.try &.downcase || "newest"
continuation = env.params.query["continuation"]? continuation = env.params.query["continuation"]?
if channel.is_age_gated if channel.is_age_gated
@ -212,7 +211,7 @@ module Invidious::Routes::API::V1::Channels
else else
begin begin
videos, next_continuation = Channel::Tabs.get_shorts( videos, next_continuation = Channel::Tabs.get_shorts(
channel, continuation: continuation, sort_by: sort_by channel, continuation: continuation
) )
rescue ex rescue ex
return error_json(500, ex) return error_json(500, ex)

View file

@ -42,9 +42,6 @@ module Invidious::Routes::API::V1::Misc
format = env.params.query["format"]? format = env.params.query["format"]?
format ||= "json" format ||= "json"
listen_param = env.params.query["listen"]?
listen = (listen_param == "true" || listen_param == "1")
if plid.starts_with? "RD" if plid.starts_with? "RD"
return env.redirect "/api/v1/mixes/#{plid}" return env.redirect "/api/v1/mixes/#{plid}"
end end
@ -88,7 +85,7 @@ module Invidious::Routes::API::V1::Misc
end end
if format == "html" if format == "html"
playlist_html = template_playlist(json_response, listen) playlist_html = template_playlist(json_response)
index, next_video = json_response["videos"].as_a.skip(1 + lookback).select { |video| !video["author"].as_s.empty? }[0]?.try { |v| {v["index"], v["videoId"]} } || {nil, nil} index, next_video = json_response["videos"].as_a.skip(1 + lookback).select { |video| !video["author"].as_s.empty? }[0]?.try { |v| {v["index"], v["videoId"]} } || {nil, nil}
response = { response = {
@ -114,9 +111,6 @@ module Invidious::Routes::API::V1::Misc
format = env.params.query["format"]? format = env.params.query["format"]?
format ||= "json" format ||= "json"
listen_param = env.params.query["listen"]?
listen = (listen_param == "true" || listen_param == "1")
begin begin
mix = fetch_mix(rdid, continuation, locale: locale) mix = fetch_mix(rdid, continuation, locale: locale)
@ -147,8 +141,10 @@ module Invidious::Routes::API::V1::Misc
json.field "authorUrl", "/channel/#{video.ucid}" json.field "authorUrl", "/channel/#{video.ucid}"
json.field "videoThumbnails" do json.field "videoThumbnails" do
json.array do
Invidious::JSONify::APIv1.thumbnails(json, video.id) Invidious::JSONify::APIv1.thumbnails(json, video.id)
end end
end
json.field "index", video.index json.field "index", video.index
json.field "lengthSeconds", video.length_seconds json.field "lengthSeconds", video.length_seconds
@ -161,7 +157,7 @@ module Invidious::Routes::API::V1::Misc
if format == "html" if format == "html"
response = JSON.parse(response) response = JSON.parse(response)
playlist_html = template_mix(response, listen) playlist_html = template_mix(response)
next_video = response["videos"].as_a.select { |video| !video["author"].as_s.empty? }[0]?.try &.["videoId"] next_video = response["videos"].as_a.select { |video| !video["author"].as_s.empty? }[0]?.try &.["videoId"]
response = { response = {

View file

@ -157,12 +157,10 @@ module Invidious::Routes::Embed
adaptive_fmts = video.adaptive_fmts adaptive_fmts = video.adaptive_fmts
if params.local if params.local
fmt_stream.each { |fmt| fmt["url"] = JSON::Any.new(HttpServer::Utils.proxy_video_url(fmt["url"].as_s)) } fmt_stream.each { |fmt| fmt["url"] = JSON::Any.new(URI.parse(fmt["url"].as_s).request_target) }
adaptive_fmts.each { |fmt| fmt["url"] = JSON::Any.new(URI.parse(fmt["url"].as_s).request_target) }
end end
# Always proxy DASH streams, otherwise youtube CORS headers will prevent playback
adaptive_fmts.each { |fmt| fmt["url"] = JSON::Any.new(HttpServer::Utils.proxy_video_url(fmt["url"].as_s)) }
video_streams = video.video_streams video_streams = video.video_streams
audio_streams = video.audio_streams audio_streams = video.audio_streams

View file

@ -194,7 +194,6 @@ module Invidious::Routes::Feeds
length_seconds: 0, length_seconds: 0,
premiere_timestamp: nil, premiere_timestamp: nil,
author_verified: false, author_verified: false,
author_thumbnail: nil,
badges: VideoBadges::None, badges: VideoBadges::None,
}) })
end end

View file

@ -304,6 +304,23 @@ module Invidious::Routes::Playlists
end end
end end
if env.params.query["action_create_playlist"]?
action = "action_create_playlist"
elsif env.params.query["action_delete_playlist"]?
action = "action_delete_playlist"
elsif env.params.query["action_edit_playlist"]?
action = "action_edit_playlist"
elsif env.params.query["action_add_video"]?
action = "action_add_video"
video_id = env.params.query["video_id"]
elsif env.params.query["action_remove_video"]?
action = "action_remove_video"
elsif env.params.query["action_move_video_before"]?
action = "action_move_video_before"
else
return env.redirect referer
end
begin begin
playlist_id = env.params.query["playlist_id"] playlist_id = env.params.query["playlist_id"]
playlist = get_playlist(playlist_id).as(InvidiousPlaylist) playlist = get_playlist(playlist_id).as(InvidiousPlaylist)
@ -318,8 +335,12 @@ module Invidious::Routes::Playlists
end end
end end
case action = env.params.query["action"]? email = user.email
when "add_video"
case action
when "action_edit_playlist"
# TODO: Playlist stub
when "action_add_video"
if playlist.index.size >= CONFIG.playlist_length_limit if playlist.index.size >= CONFIG.playlist_length_limit
if redirect if redirect
return error_template(400, "Playlist cannot have more than #{CONFIG.playlist_length_limit} videos") return error_template(400, "Playlist cannot have more than #{CONFIG.playlist_length_limit} videos")
@ -356,14 +377,12 @@ module Invidious::Routes::Playlists
Invidious::Database::PlaylistVideos.insert(playlist_video) Invidious::Database::PlaylistVideos.insert(playlist_video)
Invidious::Database::Playlists.update_video_added(playlist_id, playlist_video.index) Invidious::Database::Playlists.update_video_added(playlist_id, playlist_video.index)
when "remove_video" when "action_remove_video"
index = env.params.query["set_video_id"] index = env.params.query["set_video_id"]
Invidious::Database::PlaylistVideos.delete(index) Invidious::Database::PlaylistVideos.delete(index)
Invidious::Database::Playlists.update_video_removed(playlist_id, index) Invidious::Database::Playlists.update_video_removed(playlist_id, index)
when "move_video_before" when "action_move_video_before"
# TODO: Playlist stub # TODO: Playlist stub
when nil
return error_json(400, "Missing action")
else else
return error_json(400, "Unsupported action #{action}") return error_json(400, "Unsupported action #{action}")
end end

View file

@ -32,16 +32,24 @@ module Invidious::Routes::Subscriptions
end end
end end
if env.params.query["action_create_subscription_to_channel"]?.try &.to_i?.try &.== 1
action = "action_create_subscription_to_channel"
elsif env.params.query["action_remove_subscriptions"]?.try &.to_i?.try &.== 1
action = "action_remove_subscriptions"
else
return env.redirect referer
end
channel_id = env.params.query["c"]? channel_id = env.params.query["c"]?
channel_id ||= "" channel_id ||= ""
case action = env.params.query["action"]? case action
when "create_subscription_to_channel" when "action_create_subscription_to_channel"
if !user.subscriptions.includes? channel_id if !user.subscriptions.includes? channel_id
get_channel(channel_id) get_channel(channel_id)
Invidious::Database::Users.subscribe_channel(user, channel_id) Invidious::Database::Users.subscribe_channel(user, channel_id)
end end
when "remove_subscriptions" when "action_remove_subscriptions"
Invidious::Database::Users.unsubscribe_channel(user, channel_id) Invidious::Database::Users.unsubscribe_channel(user, channel_id)
else else
return error_json(400, "Unsupported action #{action}") return error_json(400, "Unsupported action #{action}")

View file

@ -164,13 +164,10 @@ module Invidious::Routes::VideoPlayback
env.response.headers["Access-Control-Allow-Origin"] = "*" env.response.headers["Access-Control-Allow-Origin"] = "*"
if location = resp.headers["Location"]? if location = resp.headers["Location"]?
url = Invidious::HttpServer::Utils.proxy_video_url(location, region: region) location = URI.parse(location)
location = "#{location.request_target}&host=#{location.host}#{region ? "&region=#{region}" : ""}"
if title = query_params["title"]? env.redirect location
url = "#{url}&title=#{URI.encode_www_form(title)}"
end
env.redirect url
break break
end end

View file

@ -121,12 +121,10 @@ module Invidious::Routes::Watch
adaptive_fmts = video.adaptive_fmts adaptive_fmts = video.adaptive_fmts
if params.local if params.local
fmt_stream.each { |fmt| fmt["url"] = JSON::Any.new(HttpServer::Utils.proxy_video_url(fmt["url"].as_s)) } fmt_stream.each { |fmt| fmt["url"] = JSON::Any.new(URI.parse(fmt["url"].as_s).request_target) }
adaptive_fmts.each { |fmt| fmt["url"] = JSON::Any.new(URI.parse(fmt["url"].as_s).request_target) }
end end
# Always proxy DASH streams, otherwise youtube CORS headers will prevent playback
adaptive_fmts.each { |fmt| fmt["url"] = JSON::Any.new(HttpServer::Utils.proxy_video_url(fmt["url"].as_s)) }
video_streams = video.video_streams video_streams = video.video_streams
audio_streams = video.audio_streams audio_streams = video.audio_streams
@ -243,10 +241,18 @@ module Invidious::Routes::Watch
end end
end end
case action = env.params.query["action"]? if env.params.query["action_mark_watched"]?
when "mark_watched" action = "action_mark_watched"
elsif env.params.query["action_mark_unwatched"]?
action = "action_mark_unwatched"
else
return env.redirect referer
end
case action
when "action_mark_watched"
Invidious::Database::Users.mark_watched(user, id) Invidious::Database::Users.mark_watched(user, id)
when "mark_unwatched" when "action_mark_unwatched"
Invidious::Database::Users.mark_unwatched(user, id) Invidious::Database::Users.mark_unwatched(user, id)
else else
return error_json(400, "Unsupported action #{action}") return error_json(400, "Unsupported action #{action}")

View file

@ -243,16 +243,17 @@ module Invidious::Routing
# Channels # Channels
get "/api/v1/channels/:ucid", {{namespace}}::Channels, :home get "/api/v1/channels/:ucid", {{namespace}}::Channels, :home
get "/api/v1/channels/:ucid/latest", {{namespace}}::Channels, :latest
get "/api/v1/channels/:ucid/videos", {{namespace}}::Channels, :videos
get "/api/v1/channels/:ucid/shorts", {{namespace}}::Channels, :shorts get "/api/v1/channels/:ucid/shorts", {{namespace}}::Channels, :shorts
get "/api/v1/channels/:ucid/streams", {{namespace}}::Channels, :streams get "/api/v1/channels/:ucid/streams", {{namespace}}::Channels, :streams
get "/api/v1/channels/:ucid/podcasts", {{namespace}}::Channels, :podcasts get "/api/v1/channels/:ucid/podcasts", {{namespace}}::Channels, :podcasts
get "/api/v1/channels/:ucid/releases", {{namespace}}::Channels, :releases get "/api/v1/channels/:ucid/releases", {{namespace}}::Channels, :releases
get "/api/v1/channels/:ucid/playlists", {{namespace}}::Channels, :playlists
get "/api/v1/channels/:ucid/community", {{namespace}}::Channels, :community
get "/api/v1/channels/:ucid/channels", {{namespace}}::Channels, :channels get "/api/v1/channels/:ucid/channels", {{namespace}}::Channels, :channels
get "/api/v1/channels/:ucid/search", {{namespace}}::Channels, :search
{% for route in {"videos", "latest", "playlists", "community", "search"} %}
get "/api/v1/channels/#{{{route}}}/:ucid", {{namespace}}::Channels, :{{route}}
get "/api/v1/channels/:ucid/#{{{route}}}", {{namespace}}::Channels, :{{route}}
{% end %}
# Posts # Posts
get "/api/v1/post/:id", {{namespace}}::Channels, :post get "/api/v1/post/:id", {{namespace}}::Channels, :post
@ -270,6 +271,11 @@ module Invidious::Routing
# Authenticated # Authenticated
# The notification APIs cannot be extracted yet! They require the *local* notifications constant defined in invidious.cr
#
# Invidious::Routing.get "/api/v1/auth/notifications", {{namespace}}::Authenticated, :notifications
# Invidious::Routing.post "/api/v1/auth/notifications", {{namespace}}::Authenticated, :notifications
get "/api/v1/auth/preferences", {{namespace}}::Authenticated, :get_preferences get "/api/v1/auth/preferences", {{namespace}}::Authenticated, :get_preferences
post "/api/v1/auth/preferences", {{namespace}}::Authenticated, :set_preferences post "/api/v1/auth/preferences", {{namespace}}::Authenticated, :set_preferences

View file

@ -75,7 +75,7 @@ module Invidious::Search
@type : Type = Type::All, @type : Type = Type::All,
@duration : Duration = Duration::None, @duration : Duration = Duration::None,
@features : Features = Features::None, @features : Features = Features::None,
@sort : Sort = Sort::Relevance, @sort : Sort = Sort::Relevance
) )
end end

View file

@ -47,7 +47,7 @@ module Invidious::Search
def initialize( def initialize(
params : HTTP::Params, params : HTTP::Params,
@type : Type = Type::Regular, @type : Type = Type::Regular,
@region : String? = nil, @region : String? = nil
) )
# Get the raw search query string (common to all search types). In # Get the raw search query string (common to all search types). In
# Regular search mode, also look for the `search_query` URL parameter # Regular search mode, also look for the `search_query` URL parameter

View file

@ -290,22 +290,26 @@ struct Invidious::User
end end
def from_newpipe(user : User, body : String) : Bool def from_newpipe(user : User, body : String) : Bool
Compress::Zip::File.open(IO::Memory.new(body), true) do |file| io = IO::Memory.new(body)
entry = file.entries.find { |file_entry| file_entry.filename == "newpipe.db" }
return false if entry.nil? Compress::Zip::File.open(io) do |file|
file.entries.each do |entry|
entry.open do |file_io| entry.open do |file_io|
# Ensure max size of 4MB # Ensure max size of 4MB
io_sized = IO::Sized.new(file_io, 0x400000) io_sized = IO::Sized.new(file_io, 0x400000)
begin next if entry.filename != "newpipe.db"
temp = File.tempfile(".db") do |tempfile|
tempfile = File.tempfile(".db")
begin begin
File.write(tempfile.path, io_sized.gets_to_end) File.write(tempfile.path, io_sized.gets_to_end)
rescue rescue
return false return false
end end
DB.open("sqlite3://" + tempfile.path) do |db| db = DB.open("sqlite3://" + tempfile.path)
user.watched += db.query_all("SELECT url FROM streams", as: String) user.watched += db.query_all("SELECT url FROM streams", as: String)
.map(&.lchop("https://www.youtube.com/watch?v=")) .map(&.lchop("https://www.youtube.com/watch?v="))
@ -319,10 +323,9 @@ struct Invidious::User
user.subscriptions = get_batch_channels(user.subscriptions) user.subscriptions = get_batch_channels(user.subscriptions)
Invidious::Database::Users.update_subscriptions(user) Invidious::Database::Users.update_subscriptions(user)
end
end db.close
ensure tempfile.delete
temp.delete if !temp.nil?
end end
end end
end end

View file

@ -106,7 +106,7 @@ struct Video
if formats = info.dig?("streamingData", "adaptiveFormats") if formats = info.dig?("streamingData", "adaptiveFormats")
return formats return formats
.as_a.map(&.as_h) .as_a.map(&.as_h)
.sort_by! { |f| f["width"]?.try &.as_i || f["audioTrack"]?.try { |a| a["audioIsDefault"]?.try { |v| v.as_bool ? -1 : 0 } } || 0 } .sort_by! { |f| f["width"]?.try &.as_i || 0 }
else else
return [] of Hash(String, JSON::Any) return [] of Hash(String, JSON::Any)
end end

View file

@ -36,13 +36,6 @@ def parse_related_video(related : JSON::Any) : Hash(String, JSON::Any)?
LOGGER.trace("parse_related_video: Found \"watchNextEndScreenRenderer\" container") LOGGER.trace("parse_related_video: Found \"watchNextEndScreenRenderer\" container")
if published_time_text = related["publishedTimeText"]?
decoded_time = decode_date(published_time_text["simpleText"].to_s)
published = decoded_time.to_rfc3339.to_s
else
published = nil
end
# TODO: when refactoring video types, make a struct for related videos # TODO: when refactoring video types, make a struct for related videos
# or reuse an existing type, if that fits. # or reuse an existing type, if that fits.
return { return {
@ -54,13 +47,16 @@ def parse_related_video(related : JSON::Any) : Hash(String, JSON::Any)?
"view_count" => JSON::Any.new(view_count || "0"), "view_count" => JSON::Any.new(view_count || "0"),
"short_view_count" => JSON::Any.new(short_view_count || "0"), "short_view_count" => JSON::Any.new(short_view_count || "0"),
"author_verified" => JSON::Any.new(author_verified), "author_verified" => JSON::Any.new(author_verified),
"published" => JSON::Any.new(published || ""),
} }
end end
def extract_video_info(video_id : String) def extract_video_info(video_id : String)
# Init client config for the API # Init client config for the API
client_config = YoutubeAPI::ClientConfig.new client_config = YoutubeAPI::ClientConfig.new
# Use the WEB_CREATOR when po_token is configured because it fully only works on this client
if CONFIG.po_token
client_config.client_type = YoutubeAPI::ClientType::WebCreator
end
# Fetch data from the player endpoint # Fetch data from the player endpoint
player_response = YoutubeAPI.player(video_id: video_id, params: "2AMB", client_config: client_config) player_response = YoutubeAPI.player(video_id: video_id, params: "2AMB", client_config: client_config)
@ -110,8 +106,15 @@ def extract_video_info(video_id : String)
new_player_response = nil new_player_response = nil
# Don't use Android test suite client if po_token is passed because po_token doesn't # Second try in case WEB_CREATOR doesn't work with po_token.
# work for Android test suite client. # Only trigger if reason found and po_token configured.
if reason && CONFIG.po_token
client_config.client_type = YoutubeAPI::ClientType::WebEmbeddedPlayer
new_player_response = try_fetch_streaming_data(video_id, client_config)
end
# Don't use Android client if po_token is passed because po_token doesn't
# work for Android client.
if reason.nil? && CONFIG.po_token.nil? if reason.nil? && CONFIG.po_token.nil?
# Fetch the video streams using an Android client in order to get the # Fetch the video streams using an Android client in order to get the
# decrypted URLs and maybe fix throttling issues (#2194). See the # decrypted URLs and maybe fix throttling issues (#2194). See the
@ -121,6 +124,14 @@ def extract_video_info(video_id : String)
new_player_response = try_fetch_streaming_data(video_id, client_config) new_player_response = try_fetch_streaming_data(video_id, client_config)
end end
# Last hope
# Only trigger if reason found or didn't work wth Android client.
# TvHtml5ScreenEmbed now requires sig helper for it to work but doesn't work with po_token.
if reason && CONFIG.po_token.nil?
client_config.client_type = YoutubeAPI::ClientType::TvHtml5ScreenEmbed
new_player_response = try_fetch_streaming_data(video_id, client_config)
end
# Replace player response and reset reason # Replace player response and reset reason
if !new_player_response.nil? if !new_player_response.nil?
# Preserve captions & storyboard data before replacement # Preserve captions & storyboard data before replacement
@ -224,17 +235,8 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
premiere_timestamp = microformat.dig?("liveBroadcastDetails", "startTimestamp") premiere_timestamp = microformat.dig?("liveBroadcastDetails", "startTimestamp")
.try { |t| Time.parse_rfc3339(t.as_s) } .try { |t| Time.parse_rfc3339(t.as_s) }
premiere_timestamp ||= player_response.dig?(
"playabilityStatus", "liveStreamability",
"liveStreamabilityRenderer", "offlineSlate",
"liveStreamOfflineSlateRenderer", "scheduledStartTime"
)
.try &.as_s.to_i64
.try { |t| Time.unix(t) }
live_now = microformat.dig?("liveBroadcastDetails", "isLiveNow") live_now = microformat.dig?("liveBroadcastDetails", "isLiveNow")
.try &.as_bool .try &.as_bool || false
live_now ||= video_details.dig?("isLive").try &.as_bool || false
post_live_dvr = video_details.dig?("isPostLiveDvr") post_live_dvr = video_details.dig?("isPostLiveDvr")
.try &.as_bool || false .try &.as_bool || false

View file

@ -20,7 +20,7 @@ module Invidious::Videos
def initialize( def initialize(
*, @url, @width, @height, @count, @interval, *, @url, @width, @height, @count, @interval,
@rows, @columns, @images_count, @rows, @columns, @images_count
) )
authority = /(i\d?).ytimg.com/.match!(@url.host.not_nil!)[1]? authority = /(i\d?).ytimg.com/.match!(@url.host.not_nil!)[1]?

View file

@ -128,7 +128,7 @@
<div class="top-left-overlay"> <div class="top-left-overlay">
<%- if env.get? "show_watched" -%> <%- if env.get? "show_watched" -%>
<form data-onsubmit="return_false" action="/watch_ajax?action=mark_watched&id=<%= item.id %>&referer=<%= env.get("current_page") %>" method="post"> <form data-onsubmit="return_false" action="/watch_ajax?action_mark_watched=1&id=<%= item.id %>&referer=<%= env.get("current_page") %>" method="post">
<input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>">
<button type="submit" class="pure-button pure-button-secondary low-profile" <button type="submit" class="pure-button pure-button-secondary low-profile"
data-onclick="mark_watched" data-id="<%= item.id %>"> data-onclick="mark_watched" data-id="<%= item.id %>">
@ -138,14 +138,14 @@
<%- end -%> <%- end -%>
<%- if plid_form = env.get?("add_playlist_items") -%> <%- if plid_form = env.get?("add_playlist_items") -%>
<%- form_parameters = "action=add_video&video_id=#{item.id}&playlist_id=#{plid_form}&referer=#{env.get("current_page")}" -%> <%- form_parameters = "action_add_video=1&video_id=#{item.id}&playlist_id=#{plid_form}&referer=#{env.get("current_page")}" -%>
<form data-onsubmit="return_false" action="/playlist_ajax?<%= form_parameters %>" method="post"> <form data-onsubmit="return_false" action="/playlist_ajax?<%= form_parameters %>" method="post">
<input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>">
<button type="submit" class="pure-button pure-button-secondary low-profile" <button type="submit" class="pure-button pure-button-secondary low-profile"
data-onclick="add_playlist_item" data-id="<%= item.id %>" data-plid="<%= plid_form %>"><i class="icon ion-md-add"></i></button> data-onclick="add_playlist_item" data-id="<%= item.id %>" data-plid="<%= plid_form %>"><i class="icon ion-md-add"></i></button>
</form> </form>
<%- elsif item.is_a?(PlaylistVideo) && (plid_form = env.get?("remove_playlist_items")) -%> <%- elsif item.is_a?(PlaylistVideo) && (plid_form = env.get?("remove_playlist_items")) -%>
<%- form_parameters = "action=remove_video&set_video_id=#{item.index}&playlist_id=#{plid_form}&referer=#{env.get("current_page")}" -%> <%- form_parameters = "action_remove_video=1&set_video_id=#{item.index}&playlist_id=#{plid_form}&referer=#{env.get("current_page")}" -%>
<form data-onsubmit="return_false" action="/playlist_ajax?<%= form_parameters %>" method="post"> <form data-onsubmit="return_false" action="/playlist_ajax?<%= form_parameters %>" method="post">
<input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>">
<button type="submit" class="pure-button pure-button-secondary low-profile" <button type="submit" class="pure-button pure-button-secondary low-profile"

View file

@ -1,13 +1,13 @@
<% if user %> <% if user %>
<% if subscriptions.includes? ucid %> <% if subscriptions.includes? ucid %>
<form action="/subscription_ajax?action=remove_subscriptions&c=<%= ucid %>&referer=<%= env.get("current_page") %>" method="post"> <form action="/subscription_ajax?action_remove_subscriptions=1&c=<%= ucid %>&referer=<%= env.get("current_page") %>" method="post">
<input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>">
<button data-type="unsubscribe" id="subscribe" class="pure-button pure-button-primary"> <button data-type="unsubscribe" id="subscribe" class="pure-button pure-button-primary">
<b><input style="all:unset" type="submit" value="<%= translate(locale, "Unsubscribe") %> | <%= sub_count_text %>"></b> <b><input style="all:unset" type="submit" value="<%= translate(locale, "Unsubscribe") %> | <%= sub_count_text %>"></b>
</button> </button>
</form> </form>
<% else %> <% else %>
<form action="/subscription_ajax?action=create_subscription_to_channel&c=<%= ucid %>&referer=<%= env.get("current_page") %>" method="post"> <form action="/subscription_ajax?action_create_subscription_to_channel=1&c=<%= ucid %>&referer=<%= env.get("current_page") %>" method="post">
<input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>">
<button data-type="subscribe" id="subscribe" class="pure-button pure-button-primary"> <button data-type="subscribe" id="subscribe" class="pure-button pure-button-primary">
<b><input style="all:unset" type="submit" value="<%= translate(locale, "Subscribe") %> | <%= sub_count_text %>"></b> <b><input style="all:unset" type="submit" value="<%= translate(locale, "Subscribe") %> | <%= sub_count_text %>"></b>

View file

@ -37,7 +37,7 @@
</a> </a>
<div class="top-left-overlay"><div class="watched"> <div class="top-left-overlay"><div class="watched">
<form data-onsubmit="return_false" action="/watch_ajax?action=mark_unwatched&id=<%= item %>&referer=<%= env.get("current_page") %>" method="post"> <form data-onsubmit="return_false" action="/watch_ajax?action_mark_unwatched=1&id=<%= item %>&referer=<%= env.get("current_page") %>" method="post">
<input type="hidden" name="csrf_token" value="<%= URI.encode_www_form(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= URI.encode_www_form(env.get?("csrf_token").try &.as(String) || "") %>">
<button type="submit" class="pure-button pure-button-secondary low-profile" <button type="submit" class="pure-button pure-button-secondary low-profile"
data-onclick="mark_unwatched" data-id="<%= item %>"><i class="icon ion-md-trash"></i></button> data-onclick="mark_unwatched" data-id="<%= item %>"><i class="icon ion-md-trash"></i></button>

View file

@ -37,7 +37,7 @@
<div class="pure-u-2-5"></div> <div class="pure-u-2-5"></div>
<div class="pure-u-1-5" style="text-align:right"> <div class="pure-u-1-5" style="text-align:right">
<h3 style="padding-right:0.5em"> <h3 style="padding-right:0.5em">
<form data-onsubmit="return_false" action="/subscription_ajax?action=remove_subscriptions&c=<%= channel.id %>&referer=<%= env.get("current_page") %>" method="post"> <form data-onsubmit="return_false" action="/subscription_ajax?action_remove_subscriptions=1&c=<%= channel.id %>&referer=<%= env.get("current_page") %>" method="post">
<input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>">
<input style="all:unset" type="submit" data-onclick="remove_subscription" data-ucid="<%= channel.id %>" value="<%= translate(locale, "unsubscribe") %>"> <input style="all:unset" type="submit" data-onclick="remove_subscription" data-ucid="<%= channel.id %>" value="<%= translate(locale, "unsubscribe") %>">
</form> </form>

View file

@ -29,7 +29,7 @@
</div> </div>
<div class="pure-u-1-5" style="text-align:right"> <div class="pure-u-1-5" style="text-align:right">
<h3 style="padding-right:0.5em"> <h3 style="padding-right:0.5em">
<form data-onsubmit="return_false" action="/token_ajax?action=revoke_token&session=<%= token[:session] %>&referer=<%= env.get("current_page") %>" method="post"> <form data-onsubmit="return_false" action="/token_ajax?action_revoke_token=1&session=<%= token[:session] %>&referer=<%= env.get("current_page") %>" method="post">
<input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= HTML.escape(env.get?("csrf_token").try &.as(String) || "") %>">
<input style="all:unset" type="submit" data-onclick="revoke_token" data-session="<%= token[:session] %>" value="<%= translate(locale, "revoke") %>"> <input style="all:unset" type="submit" data-onclick="revoke_token" data-session="<%= token[:session] %>" value="<%= translate(locale, "revoke") %>">
</form> </form>

View file

@ -159,7 +159,7 @@ we're going to need to do it here in order to allow for translations.
<% if user %> <% if user %>
<% playlists = Invidious::Database::Playlists.select_user_created_playlists(user.email) %> <% playlists = Invidious::Database::Playlists.select_user_created_playlists(user.email) %>
<% if !playlists.empty? %> <% if !playlists.empty? %>
<form data-onsubmit="return_false" class="pure-form pure-form-stacked" action="/playlist_ajax?action=add_video" method="post" target="_blank"> <form data-onsubmit="return_false" class="pure-form pure-form-stacked" action="/playlist_ajax" method="post" target="_blank">
<div class="pure-control-group"> <div class="pure-control-group">
<label for="playlist_id"><%= translate(locale, "Add to playlist: ") %></label> <label for="playlist_id"><%= translate(locale, "Add to playlist: ") %></label>
<select style="width:100%" name="playlist_id" id="playlist_id"> <select style="width:100%" name="playlist_id" id="playlist_id">
@ -170,6 +170,7 @@ we're going to need to do it here in order to allow for translations.
</div> </div>
<input type="hidden" name="csrf_token" value="<%= URI.encode_www_form(env.get?("csrf_token").try &.as(String) || "") %>"> <input type="hidden" name="csrf_token" value="<%= URI.encode_www_form(env.get?("csrf_token").try &.as(String) || "") %>">
<input type="hidden" name="action_add_video" value="1">
<input type="hidden" name="video_id" value="<%= video.id %>"> <input type="hidden" name="video_id" value="<%= video.id %>">
<button data-onclick="add_playlist_video" data-id="<%= video.id %>" type="submit" class="pure-button pure-button-primary"> <button data-onclick="add_playlist_video" data-id="<%= video.id %>" type="submit" class="pure-button pure-button-primary">
<b><%= translate(locale, "Add to playlist") %></b> <b><%= translate(locale, "Add to playlist") %></b>

View file

@ -67,8 +67,6 @@ private module Parsers
author_id = author_fallback.id author_id = author_fallback.id
end end
author_thumbnail = item_contents.dig?("channelThumbnailSupportedRenderers", "channelThumbnailWithLinkRenderer", "thumbnail", "thumbnails", 0, "url").try &.as_s
author_verified = has_verified_badge?(item_contents["ownerBadges"]?) author_verified = has_verified_badge?(item_contents["ownerBadges"]?)
# For live videos (and possibly recently premiered videos) there is no published information. # For live videos (and possibly recently premiered videos) there is no published information.
@ -150,7 +148,6 @@ private module Parsers
length_seconds: length_seconds, length_seconds: length_seconds,
premiere_timestamp: premiere_timestamp, premiere_timestamp: premiere_timestamp,
author_verified: author_verified, author_verified: author_verified,
author_thumbnail: author_thumbnail,
badges: badges, badges: badges,
}) })
end end
@ -582,7 +579,6 @@ private module Parsers
length_seconds: duration, length_seconds: duration,
premiere_timestamp: Time.unix(0), premiere_timestamp: Time.unix(0),
author_verified: false, author_verified: false,
author_thumbnail: nil,
badges: VideoBadges::None, badges: VideoBadges::None,
}) })
end end
@ -712,7 +708,6 @@ private module Parsers
length_seconds: duration, length_seconds: duration,
premiere_timestamp: Time.unix(0), premiere_timestamp: Time.unix(0),
author_verified: false, author_verified: false,
author_thumbnail: nil,
badges: VideoBadges::None, badges: VideoBadges::None,
}) })
end end
@ -1029,7 +1024,7 @@ end
def extract_items( def extract_items(
initial_data : InitialData, initial_data : InitialData,
author_fallback : String? = nil, author_fallback : String? = nil,
author_id_fallback : String? = nil, author_id_fallback : String? = nil
) : {Array(SearchItem), String?} ) : {Array(SearchItem), String?}
items = [] of SearchItem items = [] of SearchItem
continuation = nil continuation = nil

View file

@ -211,7 +211,7 @@ module YoutubeAPI
def initialize( def initialize(
*, *,
@client_type = ClientType::Web, @client_type = ClientType::Web,
@region = "US", @region = "US"
) )
end end
@ -300,8 +300,9 @@ module YoutubeAPI
end end
if client_config.screen == "EMBED" if client_config.screen == "EMBED"
# embedUrl https://www.google.com allow loading almost all video that are configured not embeddable
client_context["thirdParty"] = { client_context["thirdParty"] = {
"embedUrl" => "https://www.youtube.com/embed/#{video_id}", "embedUrl" => "https://www.google.com/",
} of String => String | Int64 } of String => String | Int64
end end
@ -370,7 +371,7 @@ module YoutubeAPI
browse_id : String, browse_id : String,
*, # Force the following parameters to be passed by name *, # Force the following parameters to be passed by name
params : String, params : String,
client_config : ClientConfig | Nil = nil, client_config : ClientConfig | Nil = nil
) )
# JSON Request data, required by the API # JSON Request data, required by the API
data = { data = {
@ -464,7 +465,7 @@ module YoutubeAPI
video_id : String, video_id : String,
*, # Force the following parameters to be passed by name *, # Force the following parameters to be passed by name
params : String, params : String,
client_config : ClientConfig | Nil = nil, client_config : ClientConfig | Nil = nil
) )
# Playback context, separate because it can be different between clients # Playback context, separate because it can be different between clients
playback_ctx = { playback_ctx = {
@ -557,7 +558,7 @@ module YoutubeAPI
def search( def search(
search_query : String, search_query : String,
params : String, params : String,
client_config : ClientConfig | Nil = nil, client_config : ClientConfig | Nil = nil
) )
# JSON Request data, required by the API # JSON Request data, required by the API
data = { data = {
@ -583,7 +584,7 @@ module YoutubeAPI
def get_transcript( def get_transcript(
params : String, params : String,
client_config : ClientConfig | Nil = nil, client_config : ClientConfig | Nil = nil
) : Hash(String, JSON::Any) ) : Hash(String, JSON::Any)
data = { data = {
"context" => self.make_context(client_config), "context" => self.make_context(client_config),
@ -605,7 +606,7 @@ module YoutubeAPI
def _post_json( def _post_json(
endpoint : String, endpoint : String,
data : Hash, data : Hash,
client_config : ClientConfig | Nil, client_config : ClientConfig | Nil
) : Hash(String, JSON::Any) ) : Hash(String, JSON::Any)
# Use the default client config if nil is passed # Use the default client config if nil is passed
client_config ||= DEFAULT_CLIENT_CONFIG client_config ||= DEFAULT_CLIENT_CONFIG