🎨 How to Make a Gradio Custom Component

Introduction

Gradio 4.0 will introduce Custom Components -- the ability for developers to create their own custom components and use them in Gradio apps. You can publish your components as Python packages so that other users can use them as well. Users will be able to use all of Gradio's existing functions, such as gr.Blocks, gr.Interface, API usage, themes, etc. with Custom Components.

We'll start by covering some basic Gradio concepts. If you'd like, you can skip down to the Quickstart.

Prerequisites

Besides a working familiarity with the gradio library (including Gradio Blocks), you'll also need to know some basic Python and/or TypeScript (Svelte), depending on the kind of Custom Component you are trying to create. Custom Components are created by duplicating an existing component (i.e. using it as a template) and then making changes. So there are three types of changes you could make:

Backend-only changes: Change the way that the data is processed in the backend but keep the frontend the same (example: modify the Video component to pass in video data as a numpy array instead of a filepath) - in which case you only need to know Python
Frontend-only changes:: Change the way that the component looks or behaves in the frontend but keep the data processing the same (example: modify the Model3D component to show the axes as well as light sources) - in which case you mainly need to know Svelte)
Both: (Most commonly), make changes to the backend and the frontend (example: modify the Gallery component to be able to display videos alongside images) -- in which case you need to know Python and Svelte

Even if you are making changes to the frontend, you generally don't need to know too much Svelte, particularly if you are tweaking an existing component.

Resources to learn Svelte:

Svelte's official interactive tutorial is very nice and how most of the Gradio team learned Svelte: https://learn.svelte.dev/tutorial/your-first-component
If you prefer videos: https://www.youtube.com/watch?v=LIfIRdRlD58&ab_channel=Academind

Gradio Concepts

In this section, we discuss a few important concepts when it comes to components in Gradio

Note: you can skip this section if you are familiar with the internals of the Gradio library, such as each component's preprocess and postprocess methods.

1. Interactive vs. Static

Every component in Gradio comes in a static variant, and most come in an interactive version as well. The static version is used when a component is displaying a value, and the user can NOT change that value by interacting with the Gradio user interface. The interactive version is used when the user is able to change the value by interacting with the Gradio UI.

Let's see some examples:

import gradio as gr

with gr.Blocks() as demo:
   gr.Textbox(value="Hello", interactive=True)
   gr.Textbox(value="Hello", interactive=False)

demo.launch()

Note that that this will produce two textboxes. The only difference: you'll be able to edit the value of the Gradio component on top, and you won't be able to edit the variant on the bottom (i.e. the textbox will be disabled).

Perhaps a more interesting example is with the Image component:

import gradio as gr

with gr.Blocks() as demo:
   gr.Image(interactive=True)
   gr.Image(interactive=False)

demo.launch()

The interactive version of the component is much more complex -- you can upload and edit images, draw sketches, etc. -- while the static version does not do any of that.

Not every component has an interactive version (for example, the gr.AnnotatedImage only appears as a static version since there's no way to interactively change the value of the annotations or the image. What's important to know here: Gradio will use the interactive version (if available) of a component if that component is used as the input to any event; otherwise, the static version will be used (e.g. if the component is only used as an output or to display data). For example:

import gradio as gr

with gr.Blocks() as demo:
   im1 = gr.Image()
   im2 = gr.Image()
   im1.change(lambda x:x, im1, im2)

demo.launch()

Gradio will render the top image as interactive and the bottom image as static. When you design custom components, you must accept the boolean interactive keyword in the constructor of your Python class, and in the frontend, you can create different versions of your component depending on whether this parameter is True or False.

2. The value and how it is preprocessed/postprocessed

The most important attribute of a component is its value. Every component has a value and it is this value that is typically set by the user in the frontend (if the component is interactive) or displayed to the user (if it is static). It is also this value that is sent to the backend function when a user triggers an event, or returned by the user's function e.g. at the end of a prediction.

So this value is passed around quite a bit -- but sometimes the format of the value needs to change between the frontend and backend. Take a look at this example:

import numpy as np
import gradio as gr

def sepia(input_img):
    sepia_filter = np.array([
        [0.393, 0.769, 0.189], 
        [0.349, 0.686, 0.168], 
        [0.272, 0.534, 0.131]
    ])
    sepia_img = input_img.dot(sepia_filter.T)
    sepia_img /= sepia_img.max()
    return sepia_img

demo = gr.Interface(sepia, gr.Image(shape=(200, 200)), "image")
demo.launch()

This will create a Gradio app which has an Image component as the input and the output. In the frontend, the Image component will actually upload the file to the server and send the filepath but this is converted to a numpy array before it is sent to a user's function (we chose numpy array by default because it is a common choice for machine learning engineers, though the Image component also supports other formats using the type parameter). Conversely, when the user returns a numpy array from their function, the numpy array is converted to a file so that it can be sent to the frontend and displayed by the Image component.

Where do these conversions happen? They happen in the preprocess and postprocess methods of your components. These are two important methods that you will need to write for your custom component. Taking a look at existing components' implementation will help you understand the way that these should be written.

3. The "Example Version" of a Component

Gradio apps support providing example inputs -- and these are very useful in helping users get started using your Gradio app. In gr.Interface, you can provide examples using the examples keyword, and in Blocks, you can provide examples using the special gr.Examples component.

E.g. at the bottom of this screenshot, we show an miniature example image of a cheetah that, when clicked, will populate the same image in the input Image component:

As a result, every Gradio component that can be used as an input must include an "example version", i.e. a miniature version of your component that used to display a preview of an example value.

In the frontend folder corresponding to your component, you'll see two files:

Example.svelte: this corresponds to the "example version" of your component
Index.svelte: this corresponds to the "regular version"

In the backend, you typically don't need to do anything -- unless you would like to modify the user-provided "value" of the examples to something else before it is sent to the frontend Example.svelte. If so, you can add an as_example() method to your class which takes in a single parameter (the original value) and returns the modified value. Let's see a quick example:

In the Radio.py backend, we see this as_example() function:

    def as_example(self, input_data):
        return next((c[0] for c in self.choices if c[1] == input_data), None)

Since self.choices is a list of tuples corresponding to (display_name, value), this converts the value that a user provides to the display value (or if the value is not present in self.choices, it is converted to None).

In the Radio.svelte frontend, we see this as the entire Example.svelte:

<script lang="ts">
	export let value: string;
	export let type: "gallery" | "table";
	export let selected = false;
</script>

<div
	class:table={type === "table"}
	class:gallery={type === "gallery"}
	class:selected
>
	{value}
</div>

<style>
	.gallery {
		padding: var(--size-1) var(--size-2);
	}
</style>

It simply displays the value in a <div>. What is type? It is a variable that can be either "gallery" or "table" depending on how the examples are displayed. The "gallery" form is used when the examples correspond to a single input component, whiel the "table" form is used when a user has multiple input components, and the examples need to populate all of them. You can adjust the display style depending on this variable. You can also adjust how the examples are displayed if a user "selects" a particular example by using the selected variable.

Now that we have those concepts clear, let's start building custom components 🚀

Quickstart

Installation

You will need to have:

Python 3.8+
Node.js v16.14+
pnpm 8.1+
A version of gradio that supports custom components. Currently, we recommend:

pip install gradio==3.45.0b13

The Custom Components Workflow

The Custom Components workflow consists of 3 steps: create, dev, and build.

create: creates a template for you to start developing a custom component
dev: launches a development server with a sample app & hot reloading allowing you to easily develop your custom component
build: builds the Python package corresponding to your custom component -- this makes things official!

1. `create`

Bootstrap a new template by running the following in any working directory:

gradio cc create MyComponent --template SimpleTextbox --install

Instead of MyComponent, give your component any name.

Instead of SimpleTextbox, you can use any Gradio component as a template. SimpleTextbox is actually a special component that a stripped-down version of the Textbox component that makes it particularly useful when creating your first custom component. Some other components that are good if you are starting out: SimpleDropdown or File.

2. `dev`

After the previous step, a new folder with your component's name in lower case should have been created. Open the newly-created folder in your favorite code editor and you should see the following file directory structure:

- backend/
- frontend/
- demo/
- pyproject.toml

The backend/ and frontend/ folders contain the code for the respective parts of your new component, and the demo/ folder contains the code for a sample Gradio app that we'll use to preview our component as we develop it. The pyproject.toml will be used to build our component in the next step

Start the development server by entering the new folder, and running:

gradio cc dev

You'll see several lines that are printed to the console. The most important one is the one that says;

Frontend Server (Go here): http://localhost:7861/

(The port number might be different for you). Click on that link to launch the demo app in hot reload mode. Now, you can start making changes to the backend and frontend you'll see the results reflected live in the sample app. (We'll go through a real example below).

3. `build`

Once, we're done making our changes, then press CTRL+C (or CMD+C) to exit the dev server, and type in:

gradio cc build

And you're done! You'll now have the Python package wheel that you can publish to pypi.

Example: a `RichTextbox`

Let's apply these steps to create our very own Gradio custom component: a variant of Textbox called RichTextbox that can support bold, italics, colors, and other rich text formatting.

We'll use BBCode as a convenient way to input the rich formatting that we want (e.g. Hello [b]World[/b], my name is [color=green]Abubakar[/color]) and we'll want our rich textbox to display the formatting when we are finished editing the value. Here's what it looks like:

Screen.Recording.2023-10-23.at.11.20.55.PM.online-video-cutter.com.mp4

1. `create`

Run:

gradio cc create RichTextbox --template SimpleTextbox --install

2. `dev`

A folder named richtextbox should be created in your working directory. Open this folder in your favorite code editor. Using a terminal, enter the folder and run:

gradio cc dev

This will print the address where the frontend server is running. For me, it is http://localhost:7861/ though yours may be different. Open this in your browser. You'll see a sample Gradio app that is created by running the code in the richtextbox/demo/app.py file.

This app is running in reload mode -- so if you make any changes in richtextbox/backend or richtextbox/frontend, you'll see them reflected live in your app. (Of course, you can also make changes to the demo itself!)

Let's make some changes:

Frontend changes

First, in order to display rich text, we can't use a HTML textbox. We'll need to use an editable <div>. So we replace this code (note that the existing code you see might be slightly different depending on the version of gradio you are using):

		<input
			data-testid="textbox"
			type="text"
			class="scroll-hide"
			bind:value
			bind:this={el}
			{placeholder}
			disabled={mode === "static"}
			dir={rtl ? "rtl" : "ltr"}
			on:keypress={handle_keypress}
		/>

with

		<div
			data-testid="textbox"
			contenteditable=true
			class="text-container"
			class:disabled={mode === "static"}
			bind:this={el}
			on:keypress={handle_keypress}
			role="textbox" 
			tabindex="0"
			dir={rtl ? "rtl" : "ltr"}
		>

This replaces the <input type="text"> with a <div contenteditable=true>. It also changes the class attributes and adds role and tabindex props for accessibility.

Similarly, replace

let el: HTMLTextAreaElement | HTMLInputElement;

with

let el: HTMLDivElement;

Delete this line: export let placeholder = "";.

Now, if you go back to your demo app, you'll see that the textboxes have disappeared and instead you'll have some blank space. If you click on the blank space, you'll be able to type text there, as if it was a textbox.

Now, let's add some event triggers -- while we're editing the div, we want to display the raw BBcode string. Once we stop editing the div, we want to convert it to rendered HTML. Let's update the <div> we defined earlier like this:

		<div
			data-testid="textbox"
			contenteditable=true
			class="text-container"
			class:disabled={mode === "static"}
			bind:this={el}
			on:keypress={handle_keypress}
			on:blur={handle_blur}
			on:focus={handle_focus}
			role="textbox" 
			tabindex="0"
			dir={rtl ? "rtl" : "ltr"}
		>
		{#if is_being_edited}
			{value}
		{:else}
			{@html _value}
		{/if}
		</div>

In the javscript section of your Svelte app, add this:

	let is_being_edited = false;
	let _value = "";

	async function handle_blur(): Promise<void> {
		await tick();
		if (mode === "static") {
			return;
		}
		value = el.innerText
		is_being_edited = false;
		el.innerText = "";
	}

	async function handle_focus(): Promise<void> {
		await tick();
		if (mode === "static") {
			el.blur();
			return;
		}
		is_being_edited = true;
	}

Now, if you go back to your app, you'll notice that you won't be able to edit the static RichTextbox anymore. And if you start editing the interactive RichTextbox, you'll be able to type in anything, but as soon as you click out, it'll get replaced with a blank value. That's because we never change the value of _value. We'll work on this next.

Create another file inside richtextbox/frontend/ called utils.js with this code:

// JS function to convert BBCode and HTML code - http;//coursesweb.net/javascript/
var BBCodeHTML = function() {
    var me = this;            // stores the object instance
    var token_match = /{[A-Z_]+[0-9]*}/ig;
  
    // regular expressions for the different bbcode tokens
    var tokens = {
      'URL' : '((?:(?:[a-z][a-z\\d+\\-.]*:\\/{2}(?:(?:[a-z0-9\\-._~\\!$&\'*+,;=:@|]+|%[\\dA-F]{2})+|[0-9.]+|\\[[a-z0-9.]+:[a-z0-9.]+:[a-z0-9.:]+\\])(?::\\d*)?(?:\\/(?:[a-z0-9\\-._~\\!$&\'*+,;=:@|]+|%[\\dA-F]{2})*)*(?:\\?(?:[a-z0-9\\-._~\\!$&\'*+,;=:@\\/?|]+|%[\\dA-F]{2})*)?(?:#(?:[a-z0-9\\-._~\\!$&\'*+,;=:@\\/?|]+|%[\\dA-F]{2})*)?)|(?:www\\.(?:[a-z0-9\\-._~\\!$&\'*+,;=:@|]+|%[\\dA-F]{2})+(?::\\d*)?(?:\\/(?:[a-z0-9\\-._~\\!$&\'*+,;=:@|]+|%[\\dA-F]{2})*)*(?:\\?(?:[a-z0-9\\-._~\\!$&\'*+,;=:@\\/?|]+|%[\\dA-F]{2})*)?(?:#(?:[a-z0-9\\-._~\\!$&\'*+,;=:@\\/?|]+|%[\\dA-F]{2})*)?)))',
      'LINK' : '([a-z0-9\-\./]+[^"\' ]*)',
      'EMAIL' : '((?:[\\w\!\#$\%\&\'\*\+\-\/\=\?\^\`{\|\}\~]+\.)*(?:[\\w\!\#$\%\'\*\+\-\/\=\?\^\`{\|\}\~]|&)+@(?:(?:(?:(?:(?:[a-z0-9]{1}[a-z0-9\-]{0,62}[a-z0-9]{1})|[a-z])\.)+[a-z]{2,6})|(?:\\d{1,3}\.){3}\\d{1,3}(?:\:\\d{1,5})?))',
      'TEXT' : '(.*?)',
      'SIMPLETEXT' : '([a-zA-Z0-9-+.,_ ]+)',
      'INTTEXT' : '([a-zA-Z0-9-+,_. ]+)',
      'IDENTIFIER' : '([a-zA-Z0-9-_]+)',
      'COLOR' : '([a-z]+|#[0-9abcdef]+)',
      'NUMBER'  : '([0-9]+)'
    };
  
    var bbcode_matches = [];        // matches for bbcode to html
  
    var html_tpls = [];             // html templates for html to bbcode
  
    var html_matches = [];          // matches for html to bbcode
  
    var bbcode_tpls = [];           // bbcode templates for bbcode to html
  
    /**
     * Turns a bbcode into a regular rexpression by changing the tokens into
     * their regex form
     */
    var _getRegEx = function(str) {
      var matches = str.match(token_match);
      var nrmatches = matches.length;
      var i = 0;
      var replacement = '';
  
      if (nrmatches <= 0) {
        return new RegExp(preg_quote(str), 'g');        // no tokens so return the escaped string
      }
  
      for(; i < nrmatches; i += 1) {
        // Remove {, } and numbers from the token so it can match the
        // keys in tokens
        var token = matches[i].replace(/[{}0-9]/g, '');
  
        if (tokens[token]) {
          // Escape everything before the token
          replacement += preg_quote(str.substr(0, str.indexOf(matches[i]))) + tokens[token];
  
          // Remove everything before the end of the token so it can be used
          // with the next token. Doing this so that parts can be escaped
          str = str.substr(str.indexOf(matches[i]) + matches[i].length);
        }
      }
  
      replacement += preg_quote(str);      // add whatever is left to the string
  
      return new RegExp(replacement, 'gi');
    };
  
    /**
     * Turns a bbcode template into the replacement form used in regular expressions
     * by turning the tokens in $1, $2, etc.
     */
    var _getTpls = function(str) {
      var matches = str.match(token_match);
      var nrmatches = matches.length;
      var i = 0;
      var replacement = '';
      var positions = {};
      var next_position = 0;
  
      if (nrmatches <= 0) {
        return str;       // no tokens so return the string
      }
  
      for(; i < nrmatches; i += 1) {
        // Remove {, } and numbers from the token so it can match the
        // keys in tokens
        var token = matches[i].replace(/[{}0-9]/g, '');
        var position;
  
        // figure out what $# to use ($1, $2)
        if (positions[matches[i]]) {
          position = positions[matches[i]];         // if the token already has a position then use that
        } else {
          // token doesn't have a position so increment the next position
          // and record this token's position
          next_position += 1;
          position = next_position;
          positions[matches[i]] = position;
        }
  
        if (tokens[token]) {
          replacement += str.substr(0, str.indexOf(matches[i])) + '$' + position;
          str = str.substr(str.indexOf(matches[i]) + matches[i].length);
        }
      }
  
      replacement += str;
  
      return replacement;
    };
  
    /**
     * Adds a bbcode to the list
     */
    me.addBBCode = function(bbcode_match, bbcode_tpl) {
      // add the regular expressions and templates for bbcode to html
      bbcode_matches.push(_getRegEx(bbcode_match));
      html_tpls.push(_getTpls(bbcode_tpl));
  
      // add the regular expressions and templates for html to bbcode
      html_matches.push(_getRegEx(bbcode_tpl));
      bbcode_tpls.push(_getTpls(bbcode_match));
    };
  
    /**
     * Turns all of the added bbcodes into html
     */
    me.bbcodeToHtml = function(str) {
      var nrbbcmatches = bbcode_matches.length;
      var i = 0;
  
      for(; i < nrbbcmatches; i += 1) {
        str = str.replace(bbcode_matches[i], html_tpls[i]);
      }
  
      return str;
    };
  
    /**
     * Turns html into bbcode
     */
    me.htmlToBBCode = function(str) {
      var nrhtmlmatches = html_matches.length;
      var i = 0;
  
      for(; i < nrhtmlmatches; i += 1) {
        str = str.replace(html_matches[i], bbcode_tpls[i]);
      }
  
      return str;
    }
  
    /**
     * Quote regular expression characters plus an optional character
     * taken from phpjs.org
     */
    function preg_quote (str, delimiter) {
      return (str + '').replace(new RegExp('[.\\\\+*?\\[\\^\\]$(){}=!<>|:\\' + (delimiter || '') + '-]', 'g'), '\\$&');
    }
  
    // adds BBCodes and their HTML
    me.addBBCode('[b]{TEXT}[/b]', '<strong>{TEXT}</strong>');
    me.addBBCode('[i]{TEXT}[/i]', '<em>{TEXT}</em>');
    me.addBBCode('[u]{TEXT}[/u]', '<span style="text-decoration:underline;">{TEXT}</span>');
    me.addBBCode('[s]{TEXT}[/s]', '<span style="text-decoration:line-through;">{TEXT}</span>');
    me.addBBCode('[color={COLOR}]{TEXT}[/color]', '<span style="color:{COLOR}">{TEXT}</span>');
  };

export var bbcodeParser = new BBCodeHTML();       // creates object instance of BBCodeHTML()

In Index.svelte, add this to the top of the JavaScript section:

	import {bbcodeParser} from "./utils";

and add this after you define _value:

	$: {
		_value = bbcodeParser.bbcodeToHtml(value || "");
	}

This will define _value to be the rendered (HTML) version of the raw (BBCode string) value, and update _value whenever value changes.

Now, go back to your app, and your component should mostly work! If you edit the interactive version, and type in raw BBCode, like "[b]Hello[/b]" and click out, it should be replaced with the rendered HTML.

But let's improve the styling. Go to the CSS section and remove any styling that involves the input element. Replace it with:

	.container > div.text-container
	{
		border: var(--input-border-width) solid var(--input-border-color);
		border-radius: var(--input-radius);
	}

	div.text-container {
		display: block;
		position: relative;
		outline: none !important;
		box-shadow: var(--input-shadow);
		background: var(--input-background-fill);
		padding: var(--input-padding);
		width: 100%;
		color: var(--body-text-color);
		font-weight: var(--input-text-weight);
		font-size: var(--input-text-size);
		line-height: var(--line-sm);
		border: none;
	}
	div.text-container:disabled
	{
		-webkit-text-fill-color: var(--body-text-color);
		-webkit-opacity: 1;
		opacity: 1;
	}

	div.text-container:focus {
		box-shadow: var(--input-shadow-focus);
		border-color: var(--input-border-color-focus);
	}

Now, go back to your app, and you should see a much nicer, functioning RichTextbox!

Backend changes

Because the "processing" of this component is very simple -- it just sends the raw BBCode to a user's function and presumably receives raw BBCode as output, we don't need to change the .preprocess() or .postprocess() methods. But since we support BBCode, we can choose a more exciting example_input! For example, you could replace the example_inputs function with:

    def example_inputs(self) -> Any:
        return "[b]Hello!![/b]"

3. `build`:

Now press CTRL+C (or CMD+C) in the terminal to exist the development server. Run:

gradio cc build

And you'll soon have the Python package wheel for your custom component that you can upload to pypi.

🚀🚀 And that's it! You just created your first Gradio custom component! 🚀🚀

Video Tutorial

A video tutorial is available here: https://youtu.be/6gt_CgD8r4Y, though some details have changed so please use the text above as the reference

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎨 How to Make a Gradio Custom Component

Introduction

Prerequisites

Gradio Concepts

Quickstart

Installation

The Custom Components Workflow

1. `create`

2. `dev`

3. `build`

Example: a `RichTextbox`

1. `create`

2. `dev`

3. `build`:

Video Tutorial

Clone this wiki locally

🎨 How to Make a Gradio Custom Component

Introduction

Prerequisites

Gradio Concepts

Quickstart

Installation

The Custom Components Workflow

1. create

2. dev

3. build

Example: a RichTextbox

1. create

2. dev

3. build:

Video Tutorial

Clone this wiki locally

1. `create`

2. `dev`

3. `build`

Example: a `RichTextbox`

1. `create`

2. `dev`

3. `build`: