Skip to main content

Create annotator menu applets

Applets can be used to tailor the annotator experience for a particular project. One such applet is the menu applet, which is accessible by right clicking on the image or video in the annotator. After clicking the menu option, a dialog window displays the applet and enables the user to perform customized actions for the current frame. Using the Tator REST interface, information can be modified by the applet itself via fetch calls. This tutorial will demonstrate how to create an annotator menu applet.

Applet Overview

Applets are HTML files that are loaded into an iframe. For annotator menu applets specifically, they are dynamically loaded in an modal dialog shown whenever the user clicks the corresponding right click menu option. The image below shows an example of this type of applet. The iframe in the applet dialog is the darker portion of the modal and is loaded with the selected applet.

There are two main components of the applet HTML file: the HTML file that displays the content and the JavaScript class that extends the MenuAppletElement base class. Information about the current media, frame, and selected version are passed along to a singleton of this MenuAppletElement child class. This allows developers to create customizable frame-specific annotator functions accessible to users without needing to launch a separate argo workflow.


Annotator menu applets are HTML pages that utilize the existing tatorUi library. Since the applet is loaded into an iframe, the HTML head/body code defines what is displayed to the user. Custom CSS can be defined here in the HTML file. If desired, Tator's main CSS file can be referenced with:

<link rel="stylesheet" type="text/css" href="/static/components.css"></link>

The iframe is a flex element and its styling cannot presently be altered from the applet. The dimensions of the applet dialog itself can be altered. Refer to the Applet Interface section for more information.

Applet Interface

The interface to communicate with the rest of the Tator annotator page is via a singular object whose parent class is MenuAppletElement. The applet dialog window (which resides in the annotator-page) calls functions defined by this class.

The Applet html_file must define a JavaScript class that extends MenuAppletElement. This can be accomplished by referencing the exported webpack tatorUi library:

<script src="/static/annotation.js"></script>
class NewApplet extends tatorUi.annotation.MenuAppletElement {
customElements.define("new-applet", NewApplet);

Parent Functions

There are parent functions defined in MenuAppletElement that are called by the applet dialog window. These functions have defaults, but they are expected to be overridden by the child class to provide the customized user experience. The following table outlines the functions and its role. Refer to the menu-applet-element.js source code for more information.

getModalHeightSet applet's iframe's height with a specific pixel string (e.g. "100px"). If an empty string is returned, the height is not adjusted.
getModalWidthSet the modal's width. The width of the modal cannot be adjusted to a specific width. However, three widths are provided: "default"
getModalTitleSet the modal's title
getAcceptButtonTextSet the modal's accept button's textContent
updateDataUpdate the applet with the current data. This is called when the applet is loaded (i.e. user clicks the menu option) and before updateUI is called. The following fields are set in the object's _data field: (frame, version, media, projectId). More specific information can be found in the setApplet function in menu-applet-dialog.js
updateUIExplicitly updates applet's UI. This is called when the applet is loaded (i.e. user clicks the menu option) and after the updateData function has been called.
acceptCalled when the user presses the accept button. This is effectively the last action of the applet.


To dynamically communicate with the annotator page itself, events can be dispatched from the applet. The following events are listened to by the applet dialog window.

Event.detail FieldsDescription
displayProgressMessagemessage: stringDisplays a progress message in the annotator header
displayErrorMessagemessage: stringDisplays an error message in the annotator header
displaySuccessMessagemessage: stringDisplays a success message in the annotator header
displayLoadingScreenNoneDisplays the loading screen and disables user control of the annotator. The applet must emit the hideLoadingScreen event to restore control back to the user.
hideLoadingScreenNoneHides the loading screen and reenables user control.

Required ID

It is expected that there is a single HTML element whose id is mainApplet of the newly defined applet class (example shown below). This ID is used by the dialog window to directly communicate with the instantiated applet.

<new-applet id="mainApplet"></new-applet>

Registering Applets

Applets can be registered via tator-py or through the project settings page. To specifically register an annotator right-click menu applet, its categories list must include annotator-menu. If it does not have this category, it will be not be included in the right-click menu options and may be included in other Applet lists such as the Dashboard Portal.

Note: The name of the applet is what is displayed in the right click menu.

Example Applet

The following .html file is an example annotator menu applet

<!DOCTYPE html>
<link rel="stylesheet" type="text/css" href="/static/components.css"></link>
<script src="/static/annotation.js"></script>

<body style="padding: 0px;">


class HelloWorld extends tatorUi.annotation.MenuAppletElement {
constructor() {

// Create the main div content area
this._contentDiv = document.createElement("div");
this._contentDiv.setAttribute("class", "d-flex flex-grow flex-column text-gray f2 px-4 py-2"); = "212px";

var label = document.createElement("div");
label.setAttribute("class", "h3 text-white text-semibold px-3 py-2 my-3 text-center");
label.textContent = "Hello World!";

* @returns string
getModalTitle() {
return "Hello World";

* Review menu-applet-dialog.js for more information about expected values.
* @returns string
getModalWidth() {
return "wide";

* Override parent method
accept() {
this.dispatchEvent(new CustomEvent(
"displaySuccessMessage", {
detail: {
message: "Success message!"

customElements.define("hello-world-menu-applet", HelloWorld);


<hello-world-menu-applet id="mainApplet"></hello-world-menu-applet>