Starting guide

PERFECT CORP.

CameraKit Starting Guide

This guide shows how to integrate and use PerfectLibCameraKit on iOS.

System Requirements

Minimum requirements

Prerequisites

Development environment

Additional settings

Installation

Download the CameraKit SDK package from Perfect Console.

The SDK package contains: - PerfectLibCameraKit.framework - model: the folder that contains the model files used by CameraKit

The following steps guide you through the CameraKit installation process.

1. Copy resources into your project

2. Add the framework

3. Add linker flags

4. Add required permissions

5. Add the SDK privacy manifest

Create CameraKit

Create a CameraKit instance with the path of the model folder.

CameraKit.create(withModelPath: modelPath) { [weak self] cameraKit, error in
    if let error = error {
        print("Create CameraKit failed: \(error.localizedDescription)")
        return
    }

    self?.cameraKit = cameraKit
    self?.cameraKit?.delegate = self
}

If the model files are copied into the app bundle, pass the bundle path of the model directory.

let modelPath = Bundle.main.path(forResource: "model", ofType: "")

Open Camera State

CameraKit does not open the camera device by itself. After your app configures the camera, notify CameraKit whether the active camera is front-facing.

cameraKit?.onCameraOpen(true)

Use true for the front camera and false for the back camera.

Send Camera Frames

Send camera sample buffers to CameraKit from AVCaptureVideoDataOutputSampleBufferDelegate.

func captureOutput(_ output: AVCaptureOutput,
                   didOutput sampleBuffer: CMSampleBuffer,
                   from connection: AVCaptureConnection) {
    cameraKit?.sendCameraBuffer(sampleBuffer)
}

Camera output format requirement

When configuring AVCaptureVideoDataOutput, set the pixel format type to kCVPixelFormatType_420YpCbCr8BiPlanarFullRange.

output.videoSettings = [
    kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
]

Receive Quality Results

Implement CameraKitDelegate to receive frame quality results.

extension ViewController: CameraKitDelegate {
    func cameraKit(_ cameraKit: CameraKit, checkedResult: CameraKitQualityCheck) {
        print("lighting: \(checkedResult.lightingQuality)")
        print("face area: \(checkedResult.faceAreaQuality)")
        print("face pose: \(checkedResult.facePoseQuality)")
    }
}

CameraKitQualityCheck provides: - lightingQuality - faceAreaQuality - facePoseQuality - facePoseDegree - isValid

Configure CameraKit Parameters

Use setCameraKitLevel(_:) to select a preset, and CameraKitParameterBuilder with setCameraKitOverwrite(_:) to apply custom threshold values.

Use a preset

CameraKit provides these preset modes: - strict - moderate - relaxed

cameraKit?.setCameraKitLevel(.strict)

Relaxed is applied by default when CameraKit is initialized.

Overwrite the current level

Get currentParameter from CameraKit, create a builder from that parameter, configure only the values you want to change, then call build() to get a new immutable parameter and pass it to setCameraKitOverwrite(_:).

cameraKit?.setCameraKitLevel(.moderate)

if let currentParameter = cameraKit?.currentParameter {
    let builder = currentParameter.parameterBuilder
    builder
        .setFaceSizeRatio(0.60)
        .setFaceYaw(12.0)
    do {
        let parameter = try builder.build()
        cameraKit?.setCameraKitOverwrite(parameter)
    } catch {
        print(error.localizedDescription)
    }
}

Calling setCameraKitLevel(_:) again resets the thresholds back to that level’s preset values. Any values left unset on the builder are filled from the preset for currentParameter.currentLevel when the parameter object is built. If a value is outside its valid range, build() returns an error.

You can inspect and rebuild the current state using: - cameraKit?.currentParameter - cameraKit?.currentParameter?.parameterBuilder

You can create an immutable parameter object from a builder using: - try cameraKit.currentParameter.parameterBuilder.setFaceSizeRatio(0.60).build()

The builder supports these chainable methods: - setFaceSizeRatio(_:) - setFaceYaw(_:) - setFacePitchUpper(_:) - setFacePitchLower(_:) - setLightingUpper(_:) - setLightingLower(_:)

Available overwrite parameters: - faceSizeRatio - faceYaw - facePitchUpper - facePitchLower - lightingUpper - lightingLower

The following flow is recommended when building a CameraKit screen.

  1. Configure AVCaptureSession
  2. Create CameraKit
  3. Set delegate
  4. Notify CameraKit by calling onCameraOpen(_:)
  5. Send sample buffers using sendCameraBuffer(_:)
  6. Update the UI according to CameraKitQualityCheck

Example

final class CameraKitViewController: UIViewController, AVCaptureVideoDataOutputSampleBufferDelegate, CameraKitDelegate {
    private var cameraKit: CameraKit?

    override func viewDidLoad() {
        super.viewDidLoad()

        let modelPath = Bundle.main.path(forResource: "model", ofType: "")
        CameraKit.create(withModelPath: modelPath) { [weak self] cameraKit, error in
            guard let self = self, let cameraKit = cameraKit, error == nil else {
                return
            }

            self.cameraKit = cameraKit
            self.cameraKit?.delegate = self
            self.cameraKit?.onCameraOpen(true)
            self.cameraKit?.setCameraKitLevel(.moderate)
        }
    }

    func captureOutput(_ output: AVCaptureOutput,
                       didOutput sampleBuffer: CMSampleBuffer,
                       from connection: AVCaptureConnection) {
        cameraKit?.sendCameraBuffer(sampleBuffer)
    }

    func cameraKit(_ cameraKit: CameraKit, checkedResult: CameraKitQualityCheck) {
        print(checkedResult.isValid)
    }
}

Notes