Integration¶

Overview¶

The Knomi Web SDK comes with a Web Assembly interface for web integration. This chapter will outline the requirements for integration, how to operate the developer demo program, and which parts of the demo source code corresponds to the general integration tasks outlined in the Application Design chapter.

Integration Requirements¶

The Knomi Web SDK requires a https web server to host the Web Assembly binary and interface files. It also requires a integrated or external camera to capture images. Knomi Rest servers are required, and they must be configured to accept encrypted packages. The servers require an asymmetric RSA private key, while the Knomi Web client requires a matching public key. Headers and footers are required for both public and private keys. New line characters are also required. There should be no extra characters in the string.

Encryption Key File Format¶

The Knomi Web SDK uses encryption to secure server packages. It uses an asymmetric RSA public/private key format. A bit length of 2048 or 4096 is recommended for security. The header, footer, and new line characters are required. There should be no extra characters in the string.

Public Key¶

Below is an example public key generated by OpenSSL.

Private Key¶

Below is an example private key generated by OpenSSL.

Updating Encryption Keys¶

The Knomi Web SDK has a set of example keys included, but these should not be used in production environments. When you have generated your public/private keys, you then need to update the Web Assembly binary. Only store the public key inside of the Web Assembly binary. The private key will be given to the server.

aw_knomi_web_config_tool -w KnomiWeb.wasm -p MyPublicKey.txt

Setting Encryption Key Dynamically in JavaScript¶

Instead of using the public encryption embedded in the Knomi Web WASM, you can set the public encryption in the JavaScription. For example:

const customEncryptionKey =`-----BEGIN PUBLIC KEY-----
MIGeMA0GCSqGSIb3DQEBAQUAA4GMADCBiAKBgEzS5YrXoWywb36swxN11B52dEcE
VejyHYfGofHakntlPaTdb32ABAdJ9KuLpJlviEsB/RW061BEvjdBetYrYslRr+Vz
sXbBrO36EKiJ7HwfbZta34GbnznbBHtc2WyuWbZCPZwG1EphDBe/F/c5hfZcSsEo
bwqyLOYboac3crmVAgMBAAE=
-----END PUBLIC KEY-----`;
payloadObj.SetEncryptionKey(cameraObj, customEncryptionKey);

This should be done shortly after creating the payload and camera objects and before making any calls to get payloads. To switch back to using the embedded key, set customEncryptionKey to the empty string.

Demo Installation¶

In order to access the camera, Knomi Web needs to be running in a secure context. This means the files are served from localhost, 127.0.0.1 or from a server running https. To run on https, you need a valid SSL certificate. This is an example of creating one with OpenSSL:

openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365

You must use a server capable of serving the correct MIME-type for files. Web Assembly binaries require the MIME-type application/wasm to be set.

Included here is a sample python script which creates a server capable of this.

import http.server
from http.server import HTTPServer, BaseHTTPRequestHandler
import ssl
import socketserver

PORT = 8010
Handler = http.server.SimpleHTTPRequestHandler
Handler.extensions_map= {
  '.manifest': 'text/cache-manifest',
  '.html': 'text/html',
  '.png': 'image/png',
  '.jpg': 'image/jpg',
  '.svg': 'image/svg+xml',
  '.css': 'text/css',
  '.js': 'application/x-javascript',
  '.wasm': 'application/wasm',
  '': 'application/octet-stream', # Default
}
httpd = HTTPServer(("", PORT), Handler)
httpd.socket = ssl.wrap_socket(httpd.socket, keyfile="key.pem", certfile="cert.pem", server_side=True)
httpd.serve_forever()

This script is also included in the utils folder in the installer package. Make sure you have the ssl certificate files already created before running. It is run like so:

python3 start_http_server.py

Other than this requirement, installing the demo should be as simple as copying the files to your web server.

Demo Operation¶

If running the demo on a computer, first make sure to plug in the webcamera you would like to use.

Open the demo by navigating to the location of your index html file. If this is a local server using our python script, then it would be https://localhost:8010.

Upon navigating to the page, your browser may ask you to give permissions to access the camera. Once the page loads, you should see a preview for the camera. If you do not, check that your camera is plugged in and working normally.

Select a Capture Target then enter a username in the Username field.

Press the Capture button to start capturing images. If you wish to stop the capture process, press the Stop button.

Position your face or document to fit within the region of interest overlay. Autocapture feedback should be displayed to help you position correctly.

After autocapture finishes, a popup will display giving the results.

Application Code¶

This section describes how to use the Knomi Web API in a web application.

Importing the Knomi Web library¶

To import the Knomi Web SDK, add this line to your html.

<script type="text/javascript" src="KnomiWeb.js"></script>

Adding Preview tag¶

This html tag is where the camera preview images will be drawn.

<video id="previewWindow" class="shadow bg-dark" playsinline autoplay muted>
  Your browser does not support the video tag.
</video>

Adding Region of Interest Overlay tag¶

In previous versions of KnomiWeb you could set up a canvas and KnomiWeb would draw a region of interest overlay image into it. It was decided that it would be more flexible to allow the application to generate their own region of interest overlay so this functionality has been removed from the library.

Creating the KnomiWebAllReady function¶

Next, create the KnomiWebAllReady() function in your JavaScript. This function is automatically called by KnomiWeb SDK when it and the rest of the web page is finished loading.

function KnomiWebAllReady() {
}

Creating and Initializing a Camera Object¶

A camera object is needed in order to capture images. Initial properties can be set, and then the camera is initialized. The camera tag ID is the html id where preview window will be drawn. The functions represented by FINISHED_INITIALIZING_SUCCESS_FN and FINISHED_INITIALIZING_FAILURE_FN represent the function names in your JavaScript.

function cameraFinishedInitializingSuccess() {
   ...
}

function cameraFinishedInitializingFailure(error) {
   ...
}

function KnomiWebAllReady() {
   let cameraObj = new Module.Camera();
   cameraObj.SetPropertyInt(Module.CameraProperty.ORIENTATION, cameraObj.IsMobileDevice() ? Module.CameraOrientation.PORTRAIT.value : Module.CameraOrientation.LANDSCAPE.value);
   cameraObj.SetPropertyString(Module.CameraProperty.CAMERA_TAG_ID, "previewWindow");
   cameraObj.SetPropertyString(Module.CameraProperty.FINISHED_INITIALIZING_SUCCESS_FN, "cameraFinishedInitializingSuccess");
   cameraObj.SetPropertyString(Module.CameraProperty.FINISHED_INITIALIZING_FAILURE_FN, "cameraFinishedInitializingFailure");
   cameraObj.Initialize();
}

NOTE: The camera object must available in the global namespace and named cameraObj. If the camera object is created and stored in a different place, you can satisfy the above requirement by setting a cameraObj property on the window object. For example:

window.cameraObj = myCameraObjectVariable;

If you will be recording video or audio, you will need to set a callback function that will receive the encrypted video/audio recording and any MediaRecorder error events:

function recordingAvailable(recordingData) {
  ...
}

function recordingError(event) {
  ...
}

function KnomiWebAllReady() {
  ...
  cameraObj.SetPropertyString(Module.CameraProperty.RECORDING_AVAILABLE_FN, "recordingAvailable");
  cameraObj.SetPropertyString(Module.CameraProperty.RECORDING_ERROR_FN, "recordingError");
}

Setting the Camera Resolution for Face (optional)¶

The resolution for Face capture can optionally be set by using the CameraProperty.CAMERA_VIDEO_W and CameraProperty.CAMERA_VIDEO_H properties. For example:

cameraObj.SetPropertyInt(Module.CameraProperty.CAMERA_VIDEO_W, 1080);
cameraObj.SetPropertyInt(Module.CameraProperty.CAMERA_VIDEO_H, 1440);

If only one value is set, the other value will be calculated base on a 3:4 ratio.

If the resolution isn’t set, the for mobile devices the resolution will be the default resolution when access the camera from web browser (this ensures maximum compatibility). For web cameras the resolution will be set to 1280x720.

Note

Not all cameras support all resolutions. If you set a resolution that is not supported, the finished initializing failure callback will be called with the error message Your device did not meet the minimum criteria for capturing.

Warning

We recommend sticking with the default setting on Firefox for Android. Finding an accepted resolution can be challenging and may result in browser crashes or distorted images.

Setting the Facing Mode for Face Capture (optional)¶

The facing mode for face capture can optionally be set by using the CameraProperty.FACING_MODE property. For example:

cameraObj.SetPropertyString(Module.CameraProperty.FACING_MODE, "ENVIRONMENT");

Possible values are:

DEFAULT The mode is not explicitly set. Typically defaults to the camera facing user.
USER Uses the camera facing the user. On a mobile phone, this would be the front camera.
ENVIRONMENT Uses the camera facing environment. On a mobile phone, this would be the back camera.

If the capture device doesn’t support USER or ENVIRONMENT modes, the default mode will be used.

Setting the Capture Target¶

The capture target indicates the type of capture you will be doing. See the Capture Targets section for values.

let captureTarget = "DRIVERS_LICENSE_FRONT";
cameraObj.SetPropertyString(Module.CameraProperty.CAPTURE_TARGET, captureTarget);

Collecting Camera Frames¶

To have the camera object collect a series of images, call CollectFrames(). This example also checks to see if there were too many Insufficient Lighting feedback results, and if so, increases the brightness.

async function collectCameraFrames() {
    if (insufficientLightCounter != 0 && insufficientLightCounter % 3 === 0) {
        brightnessDegree = brightnessDegree + 0.4;
        cameraObj.SetPropertyDouble(Module.CameraProperty.BRIGHTNESS, brightnessDegree);
    }

    if (insufficientLightCounter > 10) {
        return Promise.reject("Brightness has been increased too many times...");
    }

    cameraObj.CollectFrames();
}

Starting/Stopping/Pausing Camera¶

The camera has functions to start, pause, and stop preview playback.

cameraObj.Play();
cameraObj.Stop();
cameraObj.Pause();

Creating Autocapture Server Package¶

This will create a server package that can be sent to /autocaptureVideoEncrypted on the PreFace Server (face capture) or /analyzeDocumentEncrypted on the Document Analyzer Server (document capture) .

async function prepareAutocapturePayload() {
  payloadObj.SetPropertyString(Module.PayloadProperty.USERNAME, payloadUsername);
  const payload = payloadObj.GetAutocapturePayload(cameraObj);
  return payload;
}

Using a Custom Autocapture Profile¶

The default auto capture profile provides a good a balance between ease of capture and quality. However, certain use cases may require a different profile. To set a custom capture profile, you set the Module.PayloadProperty.PROFILE_XML property. For example:

async function prepareAutocapturePayload() {
  const customProfileXml = '<?xml version="1.0" encoding="UTF-8"?>  <profile version="6000000"> ... </profile>';
  payloadObj.SetPropertyString(Module.PayloadProperty.PROFILE_XML, customProfileXml);

  payloadObj.SetPropertyString(Module.PayloadProperty.USERNAME, payloadUsername);
  const payload = payloadObj.GetAutocapturePayload(cameraObj);
  return payload;
}

Creating Liveness or Validation Server Package¶

This will create a server package that can be sent to /analyzeEncrypted on the Face Liveness Server or to /analyzeDocumentEncrypted on the on the Document Validation server.

You will need to set the workflow name you are using before generating the server package.

function prepareAnalyzePayload() {
   payloadObj.SetPropertyString(Module.PayloadProperty.WORKFLOW, cameraObj.IsMobileDevice() ? Module.FOXTROT4 : Module.HOTEL4);
   let payload = payloadObj.GetAnalyzePayload(cameraObj);
   return payload;
 }

Getting Region of Interest Server Package¶

This will create a server package that can be sent to /calculateROI on the PreFace Server or the /getCaptureRegion on the document Analyzer Server.

async function prepareRegionOfInterestPayload() {
  let payloadObj = new Module.Payload();
  return payloadObj.GetRegionOfInterestPayload();
}

Drawing Region of Interest¶

The demo program contains code for drawing the region of interest overlay on face and documents captures. See the drawDocumentRegionOfInterest() and drawFaceRegionOfInterest() functions in the demo code.

Capturing Video¶

When the capture target is Video, KnomiWeb will record until the cameraObj.Stop() object is called. After this, the recording available function will be called with the recorded data. The data can then be passed to a Knomi decrypt endpoint to decrypt the data.

Below is an example of decrypted video json:

{
    "mimeType": "video/x-matroska;codecs=avc1,opus",
    "video": "GkXfo6NChoEBQveBAULygQRC84EIQoKIbWF0cm9za..."
}

Note that the browser’s default container/codecs are used for recording. The default settings are used because they are carefully chosen by the browser vendor and are optimized to deliver optimal performance and memory efficiency.

Face Checking During Recording¶

KnomiWeb also has the ability to check frames for faces while recording. Three of these frames are stored and can be sent off for liveness analysis. When the recording is complete, the the ratio of successful face check submission over the total number of face check submissions will also be returned. If no faces are found, or face checking is disabled, the returned ratio value will be 0.

To configure KnomiWeb for face checking during recording, the Module.CameraProperty.RECORDING_ANALYZE_FRAME_FN string property must be set with the name of the function to call which will submit the package to the auto capture endpoint and return the results. In addition, the Module.CameraProperty.RECORDING_ANALYZE_INTERVAL integer property can be set to specify the number of seconds between each face check.

The following is an example of configuring video recording with face check enabled with checks done 5 seconds apart.

cameraObj.SetPropertyString(
    Module.CameraProperty.RECORDING_AVAILABLE_FN,
    "recordingAvailable");
cameraObj.SetPropertyString(
    Module.CameraProperty.RECORDING_ANALYZE_FRAME_FN,
    "recordingAnalyzeFrame");
cameraObj.SetPropertyInt(
    Module.CameraProperty.RECORDING_ANALYZE_INTERVAL,
    5);
cameraObj.SetPropertyString(
    Module.CameraProperty.RECORDING_ERROR_FN,
    "recordingError");

The following is an example of a function to submit the payload to auto capture server.

/* Callback function that is called to post a recording
   frame to the face analyzer for analysis */
async function recordingAnalyzeFrame(autocapturePayload) {
    const response = await fetch(autocapturePayload, {
      method: "POST",
      headers: {
          "Content-Type": "application/json"
      },
      body: analyzePayload,
    })

    const results = await response.json();
    return results;
}

The following is an example of a function that is called when the recording is done, and will submit the

// Callback function that is called when the video or audio recording
// data is available
async function recordingAvailable(recording, facePresentRatio) {

    if (facePresentRatio > 0) {
        console.log("Face found ratio: " + facePresentRatio);

        // Get and send package to check for Liveness
        let workflow = cameraObj.IsMobileDevice() ? Module.FOXTROT4 : Module.HOTEL4
        payloadObj.SetPropertyString(Module.PayloadProperty.WORKFLOW, workflow);
        const payload = payloadObj.GetAnalyzePayload(cameraObj);
        const livenessResult = await postPayloadFetch(faceLivenessUrl, payload);

        // Log results
        if (isResultLive(livenessResult)) {
            console.log("Results is live");
        } else if (isResultSpoof(livenessResult)) {
            console.log("Results is not live!");
        }
    }
    else {
        console.log("No faces found or face checking disabled");
    }
}

Capturing Voice¶

Voice recording requires the KnomiWebProcessor.js to be loaded by the KnomiWeb library. The default path to this file is KnomiWebProcessor.js. The demo program has this file in the bin so following code change sets the `` `` to change the path:

cameraObj.SetPropertyString(Module.CameraProperty.PROCESSOR_URL, "bin/KnomiWebProcessor.js");

When capturing voice, KnomiWeb will return the data at a specified interval. This data can then be sent to /calculateVoiceLengthEncrypted endpoint to calculate how much voice is contained in the recording. When an adequate amount of voice is collected, the cameraObj.Stop() function should be called. You can change the interval between callbacks by setting the VOICE_INTERVAL integer property:

cameraObj.SetPropertyInt(Module.CameraProperty.VOICE_INTERVAL, 1);

This will set the time between callbacks to 1 second.

When a recording audio, the maximum length of the recording can be specified in seconds. Currently the default is set 180 seconds or 3 minutes. To change the value, set the VOICE_MAX_TIME integer property. For example:

cameraObj.SetPropertyInt(Module.CameraProperty.VOICE_MAX_TIME, 240);

This will set the maximum recording time to 4 minutes. After this time has elapsed, the KnomiWeb will no longer record audio or callback with audio data.

The encrypted voice data can be passed to a Knomi decrypt endpoint to decrypt the data.

Below is an example of the decrypted data:

{
    "meta_data": {
        "mimeType": "audio/wav"
    },
    "voice": {
        "voiceSamples": [
            {
                "data": "UklGRiBBCgBXQVZFZm10IBAAAA..."
            }
        ]
    }
}