PERFECT CORP.

CameraKit Starting Guide

This guide shows how to integrate and use PerfectLibCameraKit on iOS.

System Requirements

Minimum requirements

• iOS 12.0 or above
• iPhone 6 or above
• iPad Air or above

Recommended requirements

• iOS 12.0 or above
• iPhone 6, iPhone 6 Plus or higher
• iPad Air 2 or higher
• iPad mini 4 or higher
• All iPad Pro models

Prerequisites

Development environment

• Xcode 14 or higher

Additional settings

• Set Enable Bitcode to No

Installation

Download the CameraKit SDK package from Perfect Console.

The SDK package contains:

• PerfectLibCameraKit.framework
• model: the folder that contains the model files used by CameraKit

The following steps guide you through the CameraKit installation process.

1. Copy resources into your project

• Unzip the SDK package.
• Drag the model folder into your project.
• In the add file dialog, select Copy items if needed.

2. Add the framework

• Add PerfectLibCameraKit.framework to the Frameworks, Libraries, and Embedded Content section in your app target settings.

3. Add linker flags

• Add the linker flags -ObjC -lc++ -framework CoreMotion.

4. Add required permissions

• CameraKit requires camera access.
• Add the camera usage description to your app’s Info.plist.

5. Add the SDK privacy manifest

• To comply with Apple’s privacy requirements, include the SDK privacy manifest file in your project.

Create CameraKit

Create a CameraKit instance with the path of the model folder.

CameraKit.create(withModelPath: modelPath) { [weak self] cameraKit, error in
    if let error = error {
        print("Create CameraKit failed: \(error.localizedDescription)")
        return
    }

    self?.cameraKit = cameraKit
    self?.cameraKit?.delegate = self
}

If the model files are copied into the app bundle, pass the bundle path of the model directory.

let modelPath = Bundle.main.path(forResource: "model", ofType: "")

Open Camera State

CameraKit does not open the camera device by itself. After your app configures the camera, notify CameraKit whether the active camera is front-facing.

cameraKit?.onCameraOpen(true)

Use true for the front camera and false for the back camera.

Send Camera Frames

Send camera sample buffers to CameraKit from AVCaptureVideoDataOutputSampleBufferDelegate.

func captureOutput(_ output: AVCaptureOutput,
                   didOutput sampleBuffer: CMSampleBuffer,
                   from connection: AVCaptureConnection) {
    cameraKit?.sendCameraBuffer(sampleBuffer)
}

Camera output format requirement

When configuring AVCaptureVideoDataOutput, set the pixel format type to kCVPixelFormatType_420YpCbCr8BiPlanarFullRange.

output.videoSettings = [
    kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
]

Receive Quality Results

Implement CameraKitDelegate to receive frame quality results.

extension ViewController: CameraKitDelegate {
    func cameraKit(_ cameraKit: CameraKit, checkedResult: CameraKitQualityCheck) {
        print("lighting: \(checkedResult.lightingQuality)")
        print("face area: \(checkedResult.faceAreaQuality)")
        print("face pose: \(checkedResult.facePoseQuality)")
    }
}

CameraKitQualityCheck provides:

• lightingQuality
• faceAreaQuality
• facePoseQuality
• facePoseDegree
• isValid

Configure CameraKit Parameters

Use setCameraKitLevel(_:) to select a preset, and CameraKitParameterBuilder with setCameraKitOverwrite(_:) to apply custom threshold values.

Use a preset

CameraKit provides these preset modes:

• strict
• moderate
• relaxed

cameraKit?.setCameraKitLevel(.strict)

Relaxed is applied by default when CameraKit is initialized.

Overwrite the current level

Get currentParameter from CameraKit, create a builder from that parameter, configure only the values you want to change, then call build() to get a new immutable parameter and pass it to setCameraKitOverwrite(_:).

cameraKit?.setCameraKitLevel(.moderate)

if let currentParameter = cameraKit?.currentParameter {
    let builder = currentParameter.parameterBuilder
    builder
        .setFaceSizeRatio(0.60)
        .setFaceYaw(12.0)
    do {
        let parameter = try builder.build()
        cameraKit?.setCameraKitOverwrite(parameter)
    } catch {
        print(error.localizedDescription)
    }
}

Calling setCameraKitLevel(_:) again resets the thresholds back to that level’s preset values. Any values left unset on the builder are filled from the preset for currentParameter.currentLevel when the parameter object is built. If a value is outside its valid range, build() returns an error.

You can inspect and rebuild the current state using:

• cameraKit?.currentParameter
• cameraKit?.currentParameter?.parameterBuilder

You can create an immutable parameter object from a builder using:

• try cameraKit.currentParameter.parameterBuilder.setFaceSizeRatio(0.60).build()

The builder supports these chainable methods:

• setFaceSizeRatio(_:)
• setFaceYaw(_:)
• setFacePitchUpper(_:)
• setFacePitchLower(_:)
• setLightingUpper(_:)
• setLightingLower(_:)

Available overwrite parameters:

• faceSizeRatio
• faceYaw
• facePitchUpper
• facePitchLower
• lightingUpper
• lightingLower

Recommended Camera Flow

The following flow is recommended when building a CameraKit screen.

1. Configure AVCaptureSession
2. Create CameraKit
3. Set delegate
4. Notify CameraKit by calling onCameraOpen(_:)
5. Send sample buffers using sendCameraBuffer(_:)
6. Update the UI according to CameraKitQualityCheck

Example

final class CameraKitViewController: UIViewController, AVCaptureVideoDataOutputSampleBufferDelegate, CameraKitDelegate {
    private var cameraKit: CameraKit?

    override func viewDidLoad() {
        super.viewDidLoad()

        let modelPath = Bundle.main.path(forResource: "model", ofType: "")
        CameraKit.create(withModelPath: modelPath) { [weak self] cameraKit, error in
            guard let self = self, let cameraKit = cameraKit, error == nil else {
                return
            }

            self.cameraKit = cameraKit
            self.cameraKit?.delegate = self
            self.cameraKit?.onCameraOpen(true)
            self.cameraKit?.setCameraKitLevel(.moderate)
        }
    }

    func captureOutput(_ output: AVCaptureOutput,
                       didOutput sampleBuffer: CMSampleBuffer,
                       from connection: AVCaptureConnection) {
        cameraKit?.sendCameraBuffer(sampleBuffer)
    }

    func cameraKit(_ cameraKit: CameraKit, checkedResult: CameraKitQualityCheck) {
        print(checkedResult.isValid)
    }
}

Notes

• Create CameraKit only after the model files are available.
• The preview UI is managed by your app. CameraKit only consumes camera frames and returns quality results.
• If your app supports camera switching, call onCameraOpen(_:) again after changing the active camera.