aeroTAP evo API list

Introduction

aeroTAP evo API is designed to use to develop touchless interface enabled application running with aeroTAP evo Touchless Interface.
Your application can show own designed cursor by disabling aeroTAP evo's virtual cursor.

Sample code is available in /aeroTAPAPI project folder.

aeroTAP evo API requirements

You have to copy aeroTAP related files to you application folder to use aeroTAP evo API.
Below lists necessary minimum dependent modules to run aeroTAP evo,. You application can communicate with aeroTAP_CAM.DLL and share the aeroTAP evo information.

aeroTAP.exe
aeroTAP_CAM.DLL
Berldle.dll
eSPDI_DM.dll
aeroBootCheck.exe

Configure aeroTAP eov first and setting the following configuration to meet with your purpose.

Setting	Description
[Welcome] Tab Guidance OFF	Turn off the aeroTAP evo guidance display feature. You can get existence of operator via aeroTAP API. Then you can show your own guidance.
[Advanced]-Tab	Turn off the Multi-hand mode
[Advanced]Tab Click Action　NONE	If your application can control Click action, turn off aeroTAP evo's click action
[General]Tab Preview OFF	No preview window. You application can retrieve camera view.

Data definition

using namespace aerotap;

All aeroTAP API related data and APIs are defined in namespace aerotap

Color image, depth image, aeroTAP evo status data structure

Allocate the following data. TAPINFO tapinfo value provides palm tracking information from aeroTAP evo.

aeroTAP evo Palm tracking data definition
typedef struct _aeroState
{
    int nSize;    // size of data structure
    bool bActive[2];
    int nObjectType[2];
    POINT3I pos[2];
    short nPerson;
    POINT3F posG;
    POINT3F pos3D[2];
    POINT3D posHead; //
    int nGesture; //
    ushort uUIState; //
    char info[14];        // reserved
} AEROSTATE;

typedef BOOL(__cdecl *AERO_GetAeroState)(void *lpszBuf);

aeroTAP evo data structure
typedef struct _sharedMEM
{
    int nMode;
    TAPINFO tapinfo;
    int nFrame;
    BITMAPINFOHEADER bmpInfoHeader;
    unsigned char pData[1280 * 720 * 3];
    BITMAPINFOHEADER bmpInfoHeaderDepth;
    WORD pDataW[1280 * 720];
    AEROSTATE aeroState;
} SHAREDMEM;

API	Description
Retrieve aeroTAP evo status information typedef BOOL(__cdecl AERO_GetAeroState)(void lpszBuf);	You can obtain current aeroTAP evo status into AEROSTAT structure Parameter: Address of AEROSTATE data structure Return value: Success TRUE , error FALSE bActive[2] If the point is valid(TRUE)/not-valid(FALSE) bObjectType[2] Type of point 0Open Palm.1 Close Palm POINT3i pos[2] Position of palm (X,Y,Z) in current camera resolution bPerson existence of operator TRUE)/無(FALSE) posG Center position of the operator(X,Y,Z) Salce is 320x240(Left Top X=0,Y=0) aeroTAP evo can track maximum two points

Calculating Height of operator

You can caliclate the height of operator using AEROSTATE aeroState.posHead data, and position of operator's head top( X,Y,Z ) (mm).

You have to set camera position (height) and angle (To floor direction is ＋、Loop direction is－ value).
For example, if you set the camera at 200cm from the floor and set angle 10 degree to the floor, use the following code.

int nCamHeadInclined = 10; // Degree
int nCamHeadPositionY= 200; // cm

POINT3F posHead =aeroState.posHead; //obtain head top position in 3D

double inclinedR = nCamHeadInclined*M_PI/180;
//　height cm
posHead.y = nCamHeadPositionY - posHead.z*sin(inclinedR)/10 - posHead.y*cos(inclinedR) /10;
// diatance from camera cm
posHead.z = posHead.z*cos(inclinedR) / 10;

Sample code C++

AEROSTATE aeroState;

aeroState.nSize = sizeof(AEROSTATE);

       if (g_pAERO_GetAeroState && g_pAERO_GetAeroState(&aeroState))
        {
            CString sTmp;
            // Operator info
            sTmp.Format(_T("%s"), aeroState.bPerson ? _T("Exist") : _T("N/A"));
            GetDlgItem(IDC_STATIC_OPERATOR)->SetWindowTextW(sTmp);

            if (aeroState.bPerson)
                sTmp.Format(_T("x: %d , y: %d, z: %d"), (int)aeroState.posG.x, (int)aeroState.posG.y, (int)aeroState.posG.z);
            else
                sTmp = _T("N/A");
            GetDlgItem(IDC_STATIC_OP_POS)->SetWindowTextW(sTmp);

            // Palm Tracking info
            sTmp.Format(_T("%s"), aeroState.bActive[0] ? _T("Tracking") : _T("N/A"));
            GetDlgItem(IDC_STATIC_PALM)->SetWindowTextW(sTmp);

            if (aeroState.bActive[0])
                sTmp.Format(_T("x: %d , y: %d, z: %d"), (int)aeroState.pos[0].x, (int)aeroState.pos[0].y, (int)aeroState.pos[0].z);
            else
                sTmp = _T("N/A");

            GetDlgItem(IDC_STATIC_PALM_POS)->SetWindowTextW(sTmp);

Sample Unity C#

How to use sample code:

1. Create new project in Unity and save it.

2. Copy aeroTAP_CAM.dll, eSPDI_DM.dll to the project folder Assets/Plugins/x86_64.

3. Clock Unity Project Assets/Plugins/x86_64/aeroCAM_DLL, and setup Inspector（OS:Windows, CPU:x86_64 )

4. GameObject->3D Object -> Cube create BOX (any object)

5. Select New Script from Inspector の[Add Component] button, create aeroTAP from [Create and Add]

6. Edit aeroTAP.cs （Copy/past the code below)

7. Run build by Ctrl+B

*Only check build works OK

8.If build works OK, then run aeroTAP evo and run created Unity project

*You an check if Unity detecting operator , palm information.

[aeroTAP.cs]

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Runtime.InteropServices;
using System;

public class aeroTAP : MonoBehaviour
{
    string message = "";
    public struct POINT3I
    {
        public int x;
        public int y;
        public int z;
    }
    public struct POINT3F
    {
        public float x;
        public float y;
        public float z;
    }
    struct _aeroState
    {
        public int nSize; // size of this structure
        public short bActive1;
        public short bActive2;
        public int nObjectType1;
        public int nObjectType2;
        public POINT3I pos1;
        public POINT3I pos2;
        public short nPerson;
        public POINT3F posG;
        public POINT3F pos3D1;
        public POINT3F pos3D2;
        public POINT3D posHead;
        public int nGesture;
        public ushort nUIState;
        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 14)]
        public string info;      // reserved
    }

    [DllImport("aeroTAP_CAM")]
    private static extern bool AERO_GetAeroState(ref _aeroState pAeroState);

    _aeroState aeroState = new _aeroState()
    {
        bActive1 = 0,
        bPerson = 0,
        nSize = System.Runtime.InteropServices.Marshal.SizeOf(typeof(_aeroState))

    };
    // Start is called before the first frame update
    void Start()
    {
        //        aeroState.info = new char(16);
    }

    // Update is called once per frame
    void Update()
    {
        message = "API returns error";


        if (AERO_GetAeroState( ref aeroState))
        {
            if ( aeroState.bPerson!=0)
            {
                message = "Person exist";
                message += string.Format("\r\n x:{0:F} y:{1:F} z:{2:F}",aeroState.posG.x, aeroState.posG.y, aeroState.posG.z);

                message += string.Format("\r\nPalm Tacking...: {0:D}", aeroState.bActive1);
                if (aeroState.bActive1!=0)
                {
                    // ObjectType 0: Open Palm, 1:Close Palm
                    message = string.Format("\r\nObjectType: {0:D}", aeroState.nObjectType1);
                    message += string.Format("\r\nPosition: x:{0:D} y:{1:D} z:{2:D}", aeroState.pos1.x, aeroState.pos1.y, aeroState.pos1.z);
                    message += string.Format("\r\nPosition in 3D x:{0:F} y:{1:F} z:{2:F}", aeroState.pos3D1.x, aeroState.pos3D1.y, aeroState.pos3D1.z);
                    transform.position = new Vector3((aeroState.pos1.x-700)/100, -(aeroState.pos1.y-400)/100, 0);
                }
            }
            else
                message = "No Person";
        }
    }
    void OnGUI()
    {
        GUI.skin.label.fontSize = 24;
        GUI.color = Color.red;
        GUI.Label(new Rect(20, 20, 800,400), message);
    }
}