Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Livestream Thumbnails #86

Open
rolandstarke opened this issue Feb 3, 2018 · 9 comments
Open

Livestream Thumbnails #86

rolandstarke opened this issue Feb 3, 2018 · 9 comments

Comments

@rolandstarke
Copy link

rolandstarke commented Feb 3, 2018

I would like to add thumbnails for my hls live stream.

On the server i have files like

stream.m3u8
segment0.ts
thumb0.jpg
segment1.ts
thumb1.jpg
segment2.ts
thumb2.jpg

So for every segment I have a thumbnail that I want to show. Would that be possible?

@tjenkinson
Copy link
Owner

Hey @rolandstarke thanks this is similar to what I was going to suggest.

You can probably use the PLAYBACK_FRAGMENT_LOADED instead of a timeout.

https://github.com/clappr/clappr/blob/1994866b7c9bd1703c4729a5cae423c0ab3be9e6/src/playbacks/hls/hls.js#L526

Also it would be better to only remove the thumbnails that have left the stream and add the new ones, instead of clearing all of them on each updates, as this might cause the ui to flicker.

@tjenkinson
Copy link
Owner

tjenkinson commented Feb 5, 2018 via email

@rolandstarke
Copy link
Author

rolandstarke commented Feb 5, 2018

Thanks a lot. For my livestream and VOD it is working now.

<div id="video"></div>


<script src="https://cdn.jsdelivr.net/npm/clappr@0.2.86/dist/clappr.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/clappr-thumbnails-plugin@3.6.0/dist/clappr-thumbnails-plugin.js"></script>
<script>
/* global Clappr */
var ClapprThumbnailsGeneratorPlugin = Clappr.UICorePlugin.extend({

    thumbnails: [],
    isBusy: false,
    thumbnailsPlugin: null,

    markUnbusy: function () {
        this.isBusy = false;
    },

    bindEvents: function () {
        this.thumbnailsPlugin = this.core.plugins.filter(plugin => plugin.name === 'scrub-thumbnails')[0];
        if (!this.thumbnailsPlugin) {
            console.error('ClapprThumbnailsGeneratorPlugin requires ClapprThumbnailsPlugin');
            return;
        }
        this.listenTo(this.core, Clappr.Events.CORE_READY, this.bindPlaybackEvents);
    },

    bindPlaybackEvents: function () {
        var currentPlayback = this.core.getCurrentPlayback();
        if (!currentPlayback._hls) {
            console.error('ClapprThumbnailsGeneratorPlugin requires HLS Playback');
            return;
        }

        currentPlayback._hls.on('hlsLevelUpdated', this.updateThumbnails.bind(this));

        if (currentPlayback.levels && currentPlayback.levels.length > 0) {
            this.updateThumbnails();
        }
    },

    updateThumbnails: function () {
        console.log('updateThumbnails called');
        var that = this;
        var currentPlayback = that.core.getCurrentPlayback();
        var level = (currentPlayback.levels[0] || {}).level;
        if (!level || !level.details || !level.details.fragments || !level.details.fragments.map) return;

        //get thumbnail paths and times from fragments
        var newThumnails = level.details.fragments.map(function (fragment) {
            return {
                time: fragment.start,
                url: fragment.baseurl + '/../' + fragment.relurl
                    .replace('segment', 'thumb')
                    .replace('.ts', '.jpg'), // segment0.ts --> thumb0.jpg
            };
        });

        //limit the thumbnails to a maximum of 200 images
        var useEachXThumbnails = Math.ceil(newThumnails.length / 200);
        newThumnails = newThumnails.filter(function (t, index) {
            return index % useEachXThumbnails === 0;
        });

        //check if there is a change. else we can stop here
        if (that.thumbnails.length === newThumnails.length
            && that.thumbnails.every(function (t, i) { return t.time === newThumnails[i].time && t.url === newThumnails[i].url; })
        ) {
            return;
        }

        //if the thumbnail plugin is still busy loading the images from the last update stop here
        if (that.isBusy) return;
        that.isBusy = true;

        console.log('updating thumbnails');
        that.thumbnailsPlugin.removeThumbnail(that.thumbnails).then(function () {
            that.thumbnails = newThumnails;
            return that.thumbnailsPlugin.addThumbnail(that.thumbnails);
        }).catch(console.error).then(that.markUnbusy.bind(that));
    }

});
</script>
<script>
var player = new Clappr.Player({
    source: "/camera/live/stream.m3u8",
    parentId: "#video",
    autoPlay: true,
    mute: true,
    persistConfig: false, /* do not save anything in localStorage */
    plugins: {
        core: [ClapprThumbnailsPlugin, ClapprThumbnailsGeneratorPlugin],
    },
    scrubThumbnails: {
        spotlightHeight: 64,
        thumbs: [], //!IMPORTANT needs to be an array, will crash if undefined
    }
});
</script>

@tjenkinson
Copy link
Owner

Nice!

@vitordarela
Copy link

Hello everyone, I'm having problems following these steps, I'm getting the following error.
"ClapprThumbnailsGeneratorPlugin requires HLS Playback"

But when I do a console.log(currentPlayback) I see my object .. and it contains _hls

@perohu
Copy link

perohu commented May 29, 2019

I would like to add thumbnails for my hls live stream.

On the server i have files like

stream.m3u8
segment0.ts
thumb0.jpg
segment1.ts
thumb1.jpg
segment2.ts
thumb2.jpg

So for every segment I have a thumbnail that I want to show. Would that be possible?

Hello,
can you please describe, how you create thumbs for all segment on the fly?
I'm using ffmpeg to create live streams, but I don't know how to create those thumbs.
Thank you

@rolandstarke
Copy link
Author

rolandstarke commented May 29, 2019

Hello, I don't have a clever ffmpeg command.

When thumb0.jpg is requested the first time the server generates it with

ffmpeg -ss 00:00:00 -i  segment0.ts -vframes 1 -filter:v scale="-1:64" thumb0.jpg

It does not work that well on my raspberry pi, as the server needs to generate 100 thumbs on the fly the first time i visit the stream for a long time. Btw that could be an improvement for the library. Only load the thumbnails when needed. (many people probably don't hover over the timeline or when, only parts of it.)

@perohu
Copy link

perohu commented May 29, 2019

Oh, I see, thank you.

@perohu
Copy link

perohu commented May 29, 2019

Thanks a lot. For my livestream and VOD it is working now.

<div id="video"></div>


<script src="https://cdn.jsdelivr.net/npm/clappr@0.2.86/dist/clappr.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/clappr-thumbnails-plugin@3.6.0/dist/clappr-thumbnails-plugin.js"></script>
<script>
/* global Clappr */
var ClapprThumbnailsGeneratorPlugin = Clappr.UICorePlugin.extend({

    thumbnails: [],
    isBusy: false,
    thumbnailsPlugin: null,

    markUnbusy: function () {
        this.isBusy = false;
    },

    bindEvents: function () {
        this.thumbnailsPlugin = this.core.plugins.filter(plugin => plugin.name === 'scrub-thumbnails')[0];
        if (!this.thumbnailsPlugin) {
            console.error('ClapprThumbnailsGeneratorPlugin requires ClapprThumbnailsPlugin');
            return;
        }
        this.listenTo(this.core, Clappr.Events.CORE_READY, this.bindPlaybackEvents);
    },

    bindPlaybackEvents: function () {
        var currentPlayback = this.core.getCurrentPlayback();
        if (!currentPlayback._hls) {
            console.error('ClapprThumbnailsGeneratorPlugin requires HLS Playback');
            return;
        }

        currentPlayback._hls.on('hlsLevelUpdated', this.updateThumbnails.bind(this));

        if (currentPlayback.levels && currentPlayback.levels.length > 0) {
            this.updateThumbnails();
        }
    },

    updateThumbnails: function () {
        console.log('updateThumbnails called');
        var that = this;
        var currentPlayback = that.core.getCurrentPlayback();
        var level = (currentPlayback.levels[0] || {}).level;
        if (!level || !level.details || !level.details.fragments || !level.details.fragments.map) return;

        //get thumbnail paths and times from fragments
        var newThumnails = level.details.fragments.map(function (fragment) {
            return {
                time: fragment.start,
                url: fragment.baseurl + '/../' + fragment.relurl
                    .replace('segment', 'thumb')
                    .replace('.ts', '.jpg'), // segment0.ts --> thumb0.jpg
            };
        });

        //limit the thumbnails to a maximum of 200 images
        var useEachXThumbnails = Math.ceil(newThumnails.length / 200);
        newThumnails = newThumnails.filter(function (t, index) {
            return index % useEachXThumbnails === 0;
        });

        //check if there is a change. else we can stop here
        if (that.thumbnails.length === newThumnails.length
            && that.thumbnails.every(function (t, i) { return t.time === newThumnails[i].time && t.url === newThumnails[i].url; })
        ) {
            return;
        }

        //if the thumbnail plugin is still busy loading the images from the last update stop here
        if (that.isBusy) return;
        that.isBusy = true;

        console.log('updating thumbnails');
        that.thumbnailsPlugin.removeThumbnail(that.thumbnails).then(function () {
            that.thumbnails = newThumnails;
            return that.thumbnailsPlugin.addThumbnail(that.thumbnails);
        }).catch(console.error).then(that.markUnbusy.bind(that));
    }

});
</script>
<script>
var player = new Clappr.Player({
    source: "/camera/live/stream.m3u8",
    parentId: "#video",
    autoPlay: true,
    mute: true,
    persistConfig: false, /* do not save anything in localStorage */
    plugins: {
        core: [ClapprThumbnailsPlugin, ClapprThumbnailsGeneratorPlugin],
    },
    scrubThumbnails: {
        spotlightHeight: 64,
        thumbs: [], //!IMPORTANT needs to be an array, will crash if undefined
    }
});
</script>

Do you have a working live stream demo with this code?
I'm trying to make this work, but I failed. "level.details" always "undefined" in the "updateThumbnails" function.
I have to admit that I'm not an expert javascript programmer :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants