iBorderCtrl technologies

The iBorderCtrl system unifies different interdisciplinary modules and converges into an overall system to speed up the procedure of crossing the borders especially for bona fide travellers. The main modules that consist the overall solution are the following:

  • the Automatic Deception Detection System (ADDS)* performs, controls and assesses the pre-registration interview by sequencing a series of questions posed to travellers by an Avatar. ADDS quantifies the probability of deceit in interviews by analysing interviewees non-verbal micro expressions. This, coupled with an avatar, moves this novel approach to deception detection to the pre-registration phase resulting in its deployment without an impact to the time spend at the border crossing by the traveller. The avatar also allows for a consistent and controllable stimuli across interviews in terms of both the verbal and non-verbal from the direction of the avatar agent to the traveller personalized to gender and language of the traveller, reducing variability compared to human interviewers and potentially improving the accuracy of the system.  Despite the large use of biometrics on security applications including in border control with the advent of digital passports that contain fingerprints digital images and physical characteristics of individuals, a traveller with ill intentions using own documents, biomarkers would not reveal their attempted deceit. iBorderCtrl deploys well established as well as novel technologies together to collect data that will move beyond biometrics and onto biomarkers of deceit.
  • the Biometrics Module incorporating fingerprints and palm vein technologies (BIO module), for the biometric identity validation of the traveller. The BIO module is used at the Border Crossing and two different scanners are used respectively. In both cases the fingerprints and/or palm vein images are compared to the relevant information stored in databases (legacy systems in the case of fingerprints and creation of a baseline database for palm vein images) in order to assist the Border Guards to validate the traveller’s identity.
  • the Face Matching Tool (FMT), receives images of the traveller (both video type and photo) in order to create at first their biometric signature. This biometric model is compared with future images of similar type obtained from the traveller during both pre-registration phase and border crossing, in order to provide a matching score.
  • the Document Authenticity Analytics Tool (DAAT) is used both at the pre-registration and the border control procedures. DAAT is responsible for the verification procedure of travel documents which the traveller provided himself during the pre-registration procedure and the border guards scanned during the border control check. The security features of travel documents (passport, visa) are examined by DAAT against fraud characteristics in an automated way. Thus, a matching score concerning the validation of the documents authenticity is derived to facilitate the Border Guards to identify fraud characteristics.
  • the Hidden Human Detection Tool (HHD) supports the Border Guard in searching and detecting hidden people inside various vehicles (i.e. passengers attempting illegal border crossing). This tool is foreseen for the Border Crossing procedure to assist the Border Guard in the field. The functionality provided is the detection of humans hidden within vehicles such as cars or closed compartments (containers carried on trucks or train wagons).
  • the External Legacy and Social interfaces system (ELSI), is used to crosscheck the traveller’s information from legacy systems, such as SIS II, providing the necessary interoperability interfaces.
  • The Risk Based Assessment Tool (RBAT), implements a risk–assessment routine which aggregates and correlates the risks estimations received by the processing of the travellers’ data and documents supporting the decision-making of the border guard. RBAT enables a common, harmonised model for risk management and implements a systematic process to stimulate compliance and prevent and/or treat the risk of non-compliance, including risk of fraud and any other risk which appears to threaten the Authorities objectives. RBAT will also identify cases that deserve further investigation, facilitating in this way better resources allocation for the Border Managers and Agents. These risks are key to the performance of the system as they declutter the information provided to the agent by compressing all data into meaningful actionable risk scores that help the agent at the border target any follow up checks and questioning to the traveller.
  • the Integrated Border Control Analytics Tool (BCAT) enables advanced post-hoc analytics that will help identify new patterns and knowledge allowing the iBorderCtrl system to adapt quickly to new situations. At the same time, BCAT will evaluate the performance of each iBorderCtrl system and its effectiveness. BCAT will also discover key patterns in the data associated with either false accept or false rejects of travellers, which can be used for better decision making at border control. BCAT will also provide analysis on traffic data, so that it can provide the traffic history and the expected traffic for certain dates.

The two phase procedure

At pre-registration step, travellers self-report at the comfort of their own home through an on-line system that collects all relevant data helping them ensure they are fulfilling their obligations and allowing for all automated checks to take place in advance, allowing for much more computationally expensive methods to be deployed. It also provides an assessment, for the case e.g. where some aspects of human error in document preparation and collection can be detected by the traveller, to inform and provide with the opportunity to correct it prior to their crossing. Information shared with the traveller is limited to that, where a potentially criminal activity being planned would not benefit from the pre-crossing checks. While identified potentially criminal crossings are flagged to the border guards allowing them to target those individuals for further targeted evaluation. This phase enables the automation of deception detection, document authentication, external database correlation, face matching and advanced risk modelling methodologies to be deployed before the traveller even gets to the border and therefore without increasing the time the border guard spends per traveller. Deception detection is a novel approach for border control deployment applications, which if considered as self-deployed through the pre-registration phase approach, it may be able to evaluate travellers, who cannot be evaluated deterministically by other methods, effectively. Thus it may prove to be a key enabler in identifying subjects that border guards should pay special attention to during the actual crossing, and an indicator of those aspects of their travel that are suspicious.

At the border crossing stage iBorderCtrl provides key technology to the border guards both integrated to existing static installations, as well as a portable hardware platform that empowers -through technology- the border guard. At this stage all information of the traveller gathered during the pre-registration phase is now available to the Border Guard with iBorderCtrl bringing all analytic results from each technology together to identify risks to the agent that support him in both an overall evaluation of the traveller, as well as highlighting specific potential issues the agent should focus on. All documents and required information needed to cross the border are re-evaluated at the border crossing in their original hard form, however as the check has already been performed the checks are for the most part limited to validating that indeed the originals contain the same information as what was collected at pre-registration. Moreover, the biometric checks (fingerprints, palm vein face matching) take place for identity verification of the traveller. Furthermore, in case that the traveller wishes to cross the border with a vehicle, an additional check is performed to detect hidden humans inside the vehicle.

The iBorderCtrl project relies on a multitude of technical innovations that would enable a more robust and efficient land border control platform. The key objective is to employ existing and proven technologies as well as novel ones in a way to empower border agents increase the accuracy (sensitivity- ability to identify problematic crossings that should be halted, and specificity – ability to identify valid crossings) and efficiency (throughput while reducing the average per traveller cost) of border checks.

technical framework

* ADDS is based on previous developments, so-called Silent Talker [1][2]. The project has adopted this technology and is well aware about the controversy around it. This tool has undergone the following steps: A scientific foundation is achieved when a research starts from a position of ignorance and follows the scientific method to dispel that ignorance. As scientists of AI, one particular research question is “Are there non-verbal behavioural indicators of deception” and the experiments carried out collect data to support or refute the relevant hypotheses.

The work started with interviewing psychologists and reviewing the psychology literature to find a pool of candidate features to which machine learning could be applied to answer the research questions. Thus, there is not an explicit model. There is however, an overall conceptual model showing there are drivers of non-verbal behaviour that create inconsistencies with truthful NVB (detectable through machine learning) when an interviewee is deceptive. These include (but are not limited to) Arousal (including “stress” and “duping delight”), cognitive load and behaviour control.

The so far collected evidence supports the hypothesis that non-verbal behaviour can be used to detect deception at levels that are statistically significantly greater than chance. Early evidence was published in [1] and [2]. Initial iBorderCtrl work was presented in [3]. Substantial work has been done in the interim to produce a scalable system that can undergo larger scale trials through web and cloud technologies, supporting the present work.

It is also important to point out that the target for the ST component of ADDS is to move to Technology Readiness Level 7 during the programme. TRL7 means “system prototype demonstration in operational environment.”

Regarding the expected accuracy- it would be wrong to expect 100% accuracy from any AI-based deception detection technology, no matter how mature. Consequently, iBorderCtrl is designed with this in mind. iBorderCtrl is a two-stage process with many components that address various aspects of the border control procedures, and each provides its own risk estimation for the traveller. These are combined by a Risk-Based Analysis Tool which synthesises a single risk score from a weighted combination of components. Therefore, deception detection is a single component of an overall robust system.The score itself is also an indicator of risk to a human guard (human-in-the-loop principle) who makes a decision whether to pass the traveller or require a second-line interview. Consequently it is highly unlikely that an overall system of which ADDS is a part will lead to “an implementation of a pseudoscientific border control.”

The consortium is conscious about the role of artificial intelligence based systems and their role in society:

  • Organisations have been too quick to adopt techniques in the past, which have been rebutted, but then retained them purely as a deterrent.
  • There is no reason to believe that stress is an effective single indicator of deception (in fact there is no indicator that there are any effective single indicators of deception)
  • iBorderCtrl is a human in the loop system and the Border Guard will use his/her experience in making the final decision.

The project is well aware of the legal and ethical issues that might arise in the context of the developed component and in particular ADDS. Such issues are dedicated to a Work Package of the project, where issues with respect to ethics and data protection are handled in close consultation with an ethics advisor. 

During the lifespan of the project, the test of the iBorderCtrl system including ADDS, has been encapsulated so as not result to any legal effects on the data subjects when crossing the border. To achieve this, travellers are invited to voluntarily participate in a simulation of a border check using the iBorderCtrl system after they have sucessfully crossed the border with the standard procedure. Any data processing in the iBorderCtrl test pilots is based on informed consent. In the first phase of the project, ethical principle and legal safeguards relating to human-machine interaction, privacy, personal data protection and informed consent, etc., have been analysed and integrated into the design of the IborderCtrl tools. They are reviewed at each stage of the project’s development and are carried in accordance with an impact assessment.  

However, using the ADDS apart form this research project in a real border check can not be based on informed consent. A legal basis will be needed, which at present does not exist in the applicable European legal framework. It also has to be noted that iBorderCtrl as a research project aims at elaborating new technologies and whether they have the capabilities to enhance the quality of border checks for both travellers and border guards, not at implementing new policies.

[1] Rothwell, J., Bandar, Z., O'Shea, J. and McLean, D., 2006. Silent talker: a new computer‐based system for the analysis of facial cues to deception. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 20(6), pp.757-777.

[2] Rothwell, J., Bandar, Z., O’Shea, J. and McLean, D., 2007. Charting the behavioural state of a person using a backpropagation neural network. Neural Computing and Applications, 16(4-5), pp.327-339.

[3] O'Shea, J. Crockett, K. Khan, Kindynis, P. Antoniades, A.  Intelligent Deception Detection through Machine Based Interviewing, In Proceedings of IEEE International Joint conference on Artificial Neural Networks (IJCNN), July 2018, in press.