Edit Green Screen Videos

Introduction

Have you ever watched one of those programming videos where the person's face is overlayed over their computer screen? That's made possible using green screens. They're widely used especially in the media and filming industries. Even though green screens are preferred more, you can achieve the same using any solid color. I won't go over why green is more suitable compared to say blue or some other color. There are tons of youtube videos explaining that. You can check out this video or this one. That said, even though the title of this tutorial focuses on green screens, you can do the same for any video with a contrasting solid background color. In this tutorial we'll be using Cloudinary and SvelteKit. SvelteKit is to Svelte.js just as Next.js is to React.js or Nuxt.js to Vue.js.

Codesandbox

The final project can be viewed on Codesandbox.

You can find the full source code on my Github repository.

Prerequisites and setup

Before moving on further, it's important to note that working knowledge of Javascript is required for this tutorial. In addition to that, familiarity with Svelte and SvelteKit is recommended although not necessary. You can easily pick up the concepts. You also need to have Node.js and NPM installed on your development environment

DISCLAIMER: Svelte and more specifically SvelteKit, is at a very early stage and a few features are missing or require workarounds. Some notable features include file upload and loading environment variables during SSR or in the endpoint routes. You can check out this and this issues for more information and insight.

Let's first create a new [SvelteKit] project. Run the following command in your terminal

1npm init svelte@next green-screen-videos-with-cloudinary

You'll be prompted for a few more options and then the CLI will scaffold a SvelteKit starter project for you. If you want to use the same options that I did, choose Skeleton project template, No for typescript, Yes for ESLint, Yes for Prettier. Follow the steps on the terminal to change the directory into your new project and install dependencies. Finally, open the project in your favorite code editor.

Cloudinary API Keys

Cloudinary offers APIs for upload of media, optimization, transformation, and delivery of uploaded media. We need API keys to communicate with their API. Luckily, you can easily get started with a free account. Please note that resources for a free account are limited, so use the API sparingly. Create a new cloudinary account if you do not already have one and log in. Navigate to the console page. Here you'll find the Cloud name, API Key, and API Secret.

Create a new file named .env.local at the root of your project and paste the following code inside.

1VITE_CLOUDINARY_CLOUD_NAME=YOUR_CLOUD_NAME
2VITE_CLOUDINARY_API_KEY=YOUR_API_KEY
3VITE_CLOUDINARY_API_SECRET=YOUR_API_SECRET

Replace YOUR_CLOUD_NAME YOUR_API_KEY and YOUR_API_SECRET with the Cloud name, API Key and API Secret values from the console page.

Yes, these are environment variables. Now I know I mentioned that SvelteKit did not have full support for environment variables. Have a look at this Github issue to understand what I meant. It's more of an issue with Vite.js, a build tool that Svelte uses, than with SvelteKit. I'd also highly recommend you read this FAQ and this blog post before proceeding any further.

To finish up the setup, let's install the dependencies

Dependencies/Libraries used

Run the following command to install the required dependencies.

1npm run install cloudinary

Sample videos for upload

Since SvelteKit doesn't directly support file upload yet, we're going to be using static files that we have pre-downloaded. I downloaded this video for the foreground then just got a random video for the background. You can find these videos in the GitHub repo.

Getting started

Create a new folder called lib under the src folder and create a new file called cloudinary.js inside src/lib. Paste the following code inside src/lib/cloudinary.js.

1// src/lib/cloudinary.js
2
3import { v2 as cloudinary } from 'cloudinary';
4
5cloudinary.config({
6    cloud_name: import.meta.env.VITE_CLOUDINARY_CLOUD_NAME,
7    api_key: import.meta.env.VITE_CLOUDINARY_API_KEY,
8    api_secret: import.meta.env.VITE_CLOUDINARY_API_SECRET
9});
10
11const CLOUDINARY_FOLDER_NAME = 'green-screen-videos/';
12
13/**
14 * Get cloudinary upload
15 *
16 * @param {string} id
17 * @returns {Promise}
18 */
19export const handleGetCloudinaryUpload = (id) => {
20    return cloudinary.api.resource(id, {
21        type: 'upload',
22        prefix: CLOUDINARY_FOLDER_NAME,
23        resource_type: 'video'
24    });
25};
26
27/**
28 * Get cloudinary uploads
29 * @returns {Promise}
30 */
31export const handleGetCloudinaryUploads = () => {
32    return cloudinary.api.resources({
33        type: 'upload',
34        prefix: CLOUDINARY_FOLDER_NAME,
35        resource_type: 'video'
36    });
37};
38
39/**
40 * Uploads a video to cloudinary and returns the upload result
41 *
42 * @param {{path: string; transformation?:TransformationOptions;publicId?: string; folder?: boolean; }} resource
43 */
44export const handleCloudinaryUpload = (resource) => {
45    return cloudinary.uploader.upload(resource.path, {
46        // Folder to store video in
47        folder: resource.folder ? CLOUDINARY_FOLDER_NAME : null,
48        // Public id of video.
49        public_id: resource.publicId,
50        // Type of resource
51        resource_type: 'auto',
52        // Transformation to apply to the video
53        transformation: resource.transformation
54    });
55};
56
57/**
58 * Deletes resources from cloudinary. Takes in an array of public ids
59 * @param {string[]} ids
60 */
61export const handleCloudinaryDelete = (ids) => {
62    return cloudinary.api.delete_resources(ids, {
63        resource_type: 'video'
64    });
65};

At the top, we import the v2 API from the cloudinary SDK and rename it to cloudinary. We then call the .config method to initialize it with our API credentials. Note how we import the environment variables. Read this for more information on that.

CLOUDINARY_FOLDER_NAME is the name of the cloudinary folder where we want to store all our uploaded videos. Storing all our similar uploads to one folder will make it easier for us to fetch all resources in a particular folder.

handleGetCloudinaryUpload and handleGetCloudinaryUploads call the api.resource and the api.resources methods respectively. These are usually for getting either a single resource using its public id, or getting all resources in a folder. You can find more information on the two APIs in the official docs.

handleCloudinaryUpload calls the uploader.upload method on the SDK to upload a file, in this case, a video. It takes in an object that contains the path to the file and a transformation array. Read the upload api reference for more information on the options you can pass.

handleCloudinaryDelete deletes resources from cloudinary by passing an array of public IDs to the api.delete_resources method. Read more about it in the official docs.

Next, we need some API endpoints. SvelteKit uses a file-based routing system. Any file ending in a .js or a .ts extension in the src/routes folder becomes an endpoint. Read about this in the SvelteKit docs.

We want all our APIs to have the /api/ prefix so let's create a new folder called api under src/routes. Inside of src/routes/api create a folder called videos. Create two files inside src/routes/api/videos, one called [...id].js and another called index.js. The files will map to the endpoints /api/videos/ and /api/videos/:id.

Paste the following inside src/routes/api/videos/index.js

1// src/routes/api/videos/index.js
2
3import {
4    handleCloudinaryDelete,
5    handleCloudinaryUpload,
6    handleGetCloudinaryUploads
7} from '$lib/cloudinary';
8
9export async function get() {
10    try {
11        const uploads = await handleGetCloudinaryUploads();
12
13        return {
14            status: 200,
15            body: {
16                result: uploads
17            }
18        };
19    } catch (error) {
20        return {
21            status: error?.statusCode ?? 400,
22            body: {
23                error
24            }
25        };
26    }
27}
28
29export async function post() {
30    try {
31        // Path to the foreground video. This is the video with the solid background color
32        const foregroundVideoPath = 'static/videos/foreground.mp4';
33
34        // The solid background color of the foreground video
35        const foregroundChromaKeyColor = '#6adb47';
36
37        // Upload the foreground video to Cloudinary
38        const foregroundUploadResponse = await handleCloudinaryUpload({
39            path: foregroundVideoPath,
40            folder: false
41        });
42
43        // Path to the background video. This is the video that will be placed in the background
44        const backgroundVideoPath = 'static/videos/background.mp4';
45
46        // Upload the background video to Cloudinary
47        const backgroundUploadResponse = await handleCloudinaryUpload({
48            path: backgroundVideoPath,
49            folder: true,
50            transformation: [
51                {
52                    width: 500,
53                    crop: 'scale'
54                },
55                {
56                    overlay: `video:${foregroundUploadResponse.public_id}`
57                },
58                {
59                    flags: 'relative',
60                    width: '0.6',
61                    crop: 'scale'
62                },
63                {
64                    color: foregroundChromaKeyColor,
65                    effect: 'make_transparent:20'
66                },
67                {
68                    flags: 'layer_apply',
69                    gravity: 'north'
70                },
71                {
72                    duration: '15.0'
73                }
74            ]
75        });
76
77        // Delete the foreground video from Cloudinary, We don't need it anymore
78        await handleCloudinaryDelete([foregroundUploadResponse.public_id]);
79
80        return {
81            status: 200,
82            body: {
83                result: backgroundUploadResponse
84            }
85        };
86    } catch (error) {
87        return {
88            status: error?.statusCode ?? 400,
89            body: {
90                error
91            }
92        };
93    }
94}

To handle requests of a particular verb/type we need to export a function corresponding to the HTTP verb. For example, to handle GET requests we export a function called get. The only exception is the DELETE verb where we use del instead since delete is a reserved keyword. This is all covered in the docs.

get calls handleGetCloudinaryUploads to get all the resources that have been uploaded to our folder in cloudinary.

post is where the magic happens. We first need the path to both the foreground video and the background video. You can download these videos from [here](https://res.cloudinary.com/hackit-africa/video/upload/v1637232264/videos-svelte/background.mp4 and Here then save them inside static/videos folder. The foreground video is the green screen video(the video whose background we want to make transparent). We also define the background color that's in the foreground video in the foregroundChromaKeyColor variable. We then upload the foreground video to cloudinary then followed by the background video. We then use Cloudinary's transformation to overlay the previously uploaded foreground video over the background video and also make the green background transparent. There's a guide showing how to make a video transparent using cloudinary. Here's the link. Finally we delete the foreground video from cloudinary since we don't need it anymore.

Paste the following code inside src/routes/api/videos/[...id].js.

1// src/routes/api/videos/[...id].js
2
3import { handleCloudinaryDelete, handleGetCloudinaryUpload } from '$lib/cloudinary';
4
5export async function get({ params }) {
6    try {
7        const result = await handleGetCloudinaryUpload(params.id);
8
9        return {
10            status: 200,
11            body: {
12                result
13            }
14        };
15    } catch (error) {
16        return {
17            status: error?.statusCode ?? 400,
18            body: {
19                error
20            }
21        };
22    }
23}
24
25export async function del({ params }) {
26    try {
27        const result = await handleCloudinaryDelete([params.id]);
28
29        return {
30            status: 200,
31            body: {
32                result
33            }
34        };
35    } catch (error) {
36        return {
37            status: error?.statusCode ?? 400,
38            body: {
39                error
40            }
41        };
42    }
43}

We're just handling two HTTP verbs here, i.e. get and delete.

get calls handleGetCloudinaryUpload to get a specific resource using its public ID.

del passes a public id to handleCloudinaryDelete and deletes the resource with that public ID.

To understand why we named our file [...id].js instead of [id].js, have a look at the rest parameters docs. These allow us to match all paths following the api/videos/:id instead of just api/videos/:id. For example, say we have an endpoint /api/videos/folder/videoid, if we just use [id].js it will only match to /api/videos/folder.

Let's move on to the front end. For the frontend, SvelteKit also uses file-based routing. Files that are inside the src/routes directory and end in the extension .svelte are treated as pages/components. Check out this documentation on page routing.

Open src/app.html and add the following to the head.

1<style>
2    body {
3        font-family: sans-serif;
4        margin: 0;
5        padding: 0;
6    }
7
8    * {
9        box-sizing: border-box;
10        --color-primary: #0070f3;
11    }
12
13    button {
14        padding: 0 20px;
15        height: 50px;
16        border: 1px solid #ccc;
17        background-color: #ffffff;
18        font-size: 1.2rem;
19        font-weight: bold;
20        cursor: pointer;
21    }
22
23    button:disabled {
24        background-color: #cfcfcf;
25    }
26
27    button:hover:not([disabled]) {
28        color: #ffffff;
29        background-color: var(--color-primary);
30    }
31</style>

It's just some simple styles that we want to have globally. Next, create a new folder under src and call it components. This folder will hold all of our shared components. Create a new file under src/components and name it Layout.svelte. Paste the following code inside of src/components/Layout.svelte.

1<nav>
2    <ul>
3        <li><a href="/">Home</a></li>
4        <li><a href="/videos">Videos</a></li>
5    </ul>
6</nav>
7
8<main>
9    <slot />
10</main>
11
12<style>
13    nav {
14        background-color: #333333;
15        color: #fff;
16        display: flex;
17        height: 100px;
18    }
19
20    nav ul {
21        display: flex;
22        flex: 1;
23        justify-content: center;
24        align-items: center;
25        list-style: none;
26        gap: 8px;
27        margin: 0;
28        padding: 0;
29    }
30
31    nav ul li a {
32        padding: 10px 20px;
33        color: #000000;
34        display: block;
35        background-color: #ffffff;
36        text-decoration: none;
37        font-weight: bold;
38    }
39
40    nav ul li a:hover {
41        color: #ffffff;
42        background-color: var(--color-primary);
43    }
44</style>

Here we just have a nav at the top and the main element where the body of our pages will go. If you're not familiar with svelte slots and component composition, check out this tutorial from the svelte website.

Paste the following code inside src/routes/index.svelte.

1<script>
2    import Layout from '../components/Layout.svelte';
3    import { goto } from '$app/navigation';
4
5    let isLoading = false;
6
7    async function onGenerate() {
8        try {
9            isLoading = true;
10
11            const response = await fetch('/api/videos', {
12                method: 'post'
13            });
14            const data = await response.json();
15            if (!response.ok) {
16                throw data;
17            }
18
19            goto('/videos/', { replaceState: false });
20        } catch (error) {
21            console.error(error);
22        } finally {
23            isLoading = false;
24        }
25    }
26</script>
27
28<Layout>
29    <div class="wrapper">
30        {#if isLoading}
31            <div class="loading">
32                <i>Loading. Please be patient.</i>
33                <hr />
34            </div>
35        {/if}
36
37        <h1>Make green screen videos transparent using Cloudinary + Svelte</h1>
38
39        <p>
40            While we're referring to them as green screen videos, you can really do this with any video
41            that has a solid background color.
42        </p>
43
44        <p>
45            You can change the background and foreground videos by editing the videos inside
46            /static/videos/
47        </p>
48
49        <p>Tap on the button below to edit the files in /static/videos/</p>
50
51        <button on:click|preventDefault={onGenerate} disabled={isLoading}>GENERATE VIDEO</button>
52
53        <br />
54        <p>or</p>
55        <br />
56        <a href="/videos">View generated videos</a>
57    </div>
58</Layout>
59
60<style>
61    div.loading {
62        color: var(--color-primary);
63    }
64    div.wrapper {
65        min-height: 100vh;
66        width: 100%;
67        display: flex;
68        flex-flow: column;
69        justify-content: center;
70        align-items: center;
71        background-color: #ffffff;
72    }
73</style>

When the user taps on the button, the onGenerate function makes a POST request to the /api/videos endpoint that we created earlier. This uploads the two videos then navigate to the videos page that we'll create shortly. I won't go much into the syntax of svelte components since that's something you can easily grasp from the svelte documentation and other tutorials.

Next, we need pages to display all videos and specific videos. Create a new folder called videos under src/routes. Please note that this is a different videos folder from the one inside the api folder. Then create two files under src/routes/videos, one called index.svelte and another [...id].svelte.

Paste the following code inside of src/routes/videos/index.svelte

1<script>
2    import Layout from '../../components/Layout.svelte';
3    import { onMount } from 'svelte';
4
5    let isLoading = false;
6    let videos = [];
7
8    onMount(async () => {
9        try {
10            isLoading = true;
11
12            const response = await fetch('/api/videos', {
13                method: 'GET'
14            });
15
16            const data = await response.json();
17
18            if (!response.ok) {
19                throw data;
20            }
21
22            videos = data.result.resources;
23        } catch (error) {
24            console.error(error);
25        } finally {
26            isLoading = false;
27        }
28    });
29</script>
30
31<Layout>
32    {#if videos.length > 0}
33        <div class="wrapper">
34            <div class="videos">
35                {#each videos as video (video.public_id)}
36                    <div class="video">
37                        <div class="thumbnail">
38                            <img src={video.secure_url.replace('.mp4', '.jpg')} alt={video.secure_url} />
39                        </div>
40
41                        <div class="actions">
42                            <a href={`/videos/${video.public_id}`}>Open Video</a>
43                        </div>
44                    </div>
45                {/each}
46            </div>
47        </div>
48    {:else}
49        <div class="no-videos">
50            <b>No videos yet</b>
51            <br />
52            <a href="/">Generate video</a>
53        </div>
54    {/if}
55
56    {#if isLoading && videos.length === 0}
57        <div class="loading">
58            <b>Loading...</b>
59        </div>
60    {/if}
61</Layout>
62
63<style>
64    div.wrapper div.videos {
65        display: flex;
66        flex-flow: row wrap;
67        gap: 20px;
68        padding: 20px;
69    }
70
71    div.wrapper div.videos div.video {
72        flex: 0 1 400px;
73        border: #ccc 1px solid;
74        border-radius: 5px;
75    }
76
77    div.wrapper div.videos div.video div.thumbnail {
78        width: 100%;
79    }
80
81    div.wrapper div.videos div.video div.thumbnail img {
82        width: 100%;
83    }
84
85    div.wrapper div.videos div.video div.actions {
86        padding: 10px;
87    }
88
89    div.loading,
90    div.no-videos {
91        height: calc(100vh - 100px);
92        display: flex;
93        flex-flow: column;
94        align-items: center;
95        justify-content: center;
96    }
97</style>

Here we're using onMount to make a GET request to the /api/videos endpoint when the component mounts. Read about onMount here. For the video thumbnails, all we have to do is replace the .mp4 extension with .jpg and Cloudinary automatically gives us a thumbnail of that video. Clicking on a video takes you to the /videos/:id page that we'll be creating next.

Paste the following code inside src/routes/videos/[...id].svelte

1<script context="module">
2    export async function load({ page, fetch }) {
3        try {
4            const response = await fetch(`/api/videos/${page.params.id}`, {
5                method: 'GET'
6            });
7
8            const data = await response.json();
9
10            if (!response.ok) {
11                throw data;
12            }
13
14            return {
15                props: {
16                    video: data.result
17                }
18            };
19        } catch (error) {
20            return {
21                status: error?.statusCode ?? 400,
22                error: error?.message ?? 'Something went wrong'
23            };
24        }
25    }
26</script>
27
28<script>
29    import Layout from '../../components/Layout.svelte';
30    import { goto } from '$app/navigation';
31
32    let isLoading = false;
33    export let video;
34
35    async function deleteVideo() {
36        try {
37            isLoading = true;
38            const response = await fetch(`/api/videos/${video.public_id}`, {
39                method: 'DELETE'
40            });
41
42            const data = await response.json();
43
44            if (!response.ok) {
45                throw data;
46            }
47
48            goto('/videos', { replaceState: true });
49        } catch (error) {
50            console.error(error);
51        } finally {
52            isLoading = false;
53        }
54    }
55</script>
56
57<Layout>
58    <div class="wrapper">
59        <div class="video">
60            <video src={video.secure_url} controls>
61                <track kind="captions" />
62            </video>
63            <div class="actions">
64                <button on:click|preventDefault={deleteVideo} disabled={isLoading}>Delete</button>
65            </div>
66        </div>
67    </div>
68</Layout>
69
70<style>
71    div.wrapper {
72        min-width: 100vh;
73        display: flex;
74        justify-content: center;
75        align-items: center;
76    }
77
78    div.wrapper div.video {
79        width: 80%;
80    }
81
82    div.wrapper div.video video {
83        width: 100%;
84        object-fit: fill;
85    }
86</style>

If you've used Next.js before, you probably know about getStaticProps or getServerSideProps. The load function that we've used here is similar to those. In SvelteKit, however, the load method is called on both Server Side Rendering and Client-Side Rendering. There are a few things you should be wary of when using load. I highly recommend you have a read of these docs.

In this case, we use load to make GET a call to the /api/videos/:id endpoint to get the video with that specific ID. That's about it for the front end. We need one more thing, a custom error page.

Create a file called __error.svelte under src/routes and paste the following code inside.

1<script>
2    import Layout from '../components/Layout.svelte';
3
4    function tryAgain() {
5        window.location.reload();
6    }
7</script>
8
9<Layout>
10    <div class="wrapper">
11        <b>Something went wrong</b>
12        <br />
13        <button on:click|preventDefault={tryAgain}>Try Again</button>
14    </div>
15</Layout>
16
17<style>
18    .wrapper {
19        display: flex;
20        flex-direction: column;
21        align-items: center;
22        justify-content: center;
23        height: calc(100vh - 100px);
24    }
25</style>

And that's it for this tutorial. You can run your app by running the following in your terminal.

1npm run dev -- --open