Introduction

The currently created robotic systems are primarily designed for replacing lost motor functions, including those after injuries, amputations, and cerebral accidents. The components of portable exoskeletons are similar to limbs, and perform kinematic tasks thereof. These technological solutions are widely used in various fields, as well as for increasing physical capacity, studying the neuro-motor monitoring, and rehabilitation purposes. The first prototype of the exoskeleton-man interface was the operator who was dubbing man’s movement and operating the machine. The second generation of interfaces for exoskeletons used for medical purposes was made as robotic devices with button control that transmited user’s commands to the motor mechanisms. The latest generation of exoskeleton interfaces is based on picking an EEG signal. Today, such interfaces are mainly used for medical purposes for people with lower limbs paralysis or walking weakness in patients after cerebral accidents, multiocular sclerosis, amyotrophic lateral sclerosis and other neurological diseases, being not only a great alternative to wheelchairs, but also making it possible to return a disabled to every-day and social life. The brain-computer interface is a convenient and direct method of sending commands to the mechanical device.

Today, scientists have approached resolving the unique challenge, i.e. connecting external electro-mechanical actuators to the brain. The idea is that the brain is directly connected to external actuators using a special system (the neurointerface) that records the activity of the nerve cells in the motor cerebral cortex and decrypts man’s intention to perform a particular movement. This will make it possible for scientists to expand understanding of the nervous system physiology, for doctors – to expand the scope of their manipulation, and for the disabled – to obtain functional replacement of missing limbs, or to compensate for the loss of their motor function. With that, the recording electrodes may either be implanted into the brain, or be fixed on the head, as it is done in picking an electroencephalogram. The systems of bilateral information exchange (input/output), direct computer communication with the brain are the ultimate goal of the research and clinical use, and will be the key for application in the future.

Currently the following physical indicators of human biological systems varying with time are being actively studied, which can be used to control machines or systems [1]: electromyography (EMG), electrooculography (EOG), cardiac electrical activity (ECG), but the electrical activity of the brain (EEG) is of the greatest interest. It is assumed that registration of brain potential that regulates functioning of the whole organism followed by mathematical modeling of the motor function will make it possible to meet the medico-social needs.

The brain-machine interface (BMI) may be defined as any system that is capable of tracking brain activity and transforming man’s intention into commands to external devices instead of sending them to effectors of the skeletal muscles [2,3]. An article recently published in the Lancet journal showed high efficiency of using BMI in neuroprostheses control systems for persons with pamplegia and relatively preserved sensory, emotional and cognitive abilities [3]. This work has been the inspiration for many scientific groups in the world for developing and improving BMIs recording brain bio-electric potentials.

Recording brain bio-electric potentials

Invasive and noninvasive BMIs are used for recording brain bio-electric potentials. There are three types of invasive interfaces: electrocorticography (ECoG), recording local field potentials (LFP) and recording spikes [4]. The ECoG signals are picked from the brain cortex, but the electrodes are not embedded into the cortex, unlike LFP and spikes recording.

In case of ECoG, the electrodes are placed directly on the exposed surface of the cerebral cortex to register its electrical activity. Compared to the intracortical electrodes, ECoG has the advantage of lower clinical risk and long-term stability. Compared to EEG, better spatial resolution is ensured, as well as bandwidth, and signal-to-noise ratio and weaker attenuation at high frequencies. Therefore, out of all possible methods of picking the signal directly from the cortex, ECoG is used more often, in particular to control prostheses of paralyzed patients [5, 6]. An important advantage of ECoG is distinguishing between the movements of the fingers, since neural mapping of the fingers is widely distributed across the cortex [7]. The signals of the cerebral cortex associated with the movements of the fingers and grip may be used to control various movements of the prosthesis. It is also possible to use the ECoG signals that are not directly related the movements of the hands, for example, tongue movements. Fifer M. et al (2012) showed successful recognition of complex movements in the ECoG signal of a patient, and defined five directions of ECoG-based control systems development [8]: improving the resolution of electrode arrays, improving the decoding algorithms, providing proprioceptive and tactile feedback, developing fully implantable ECoG systems, and solving the ethical issues.

Intracortical neural signals received from the cortex via the invasive electrodes may be used for interaction with a hand prosthesis [3, 9]. Hochberg L. et al. (2012), in course of recording neural activity in the excitable area of the cortex found that the peak patterns of planned movement remained even three years after spinal cord injury in patients with pamplegia. More over  such patients remained capable of using  the intracortical neural pulses to perform the movements of reaching out to an object and gripping in the three-dimensional space without difficult training [9]. J. L. Collinger et al. (2013) had implanted two arrays into the left part of the patient’s excitable area of the cortex, and after the training, the patient was able to use a robotic limb for coordinated and precise movements of reaching out to the subject and gripping [3].

Electrical and the magnetic activity is non-invasively measured at the surface of the head using EEG, magnetoencephalography, and other methods [10]. Unlike invasive methods, the non-invasive BMIs are not prone to risks, but the signals feature lower spatial resolution, and may be susceptible to artifacts [11, 12].

EEG is the most popular BMI, due to its low cost, high availability, mobility and safety. However, ensuring sustainable control of the prosthesis remains a difficult task, and most publications are devoted to discernment of limbs movements. Scientists have proposed an EEG-based approach for creating decoders, which can reconstruct angles of fingers joints for a movement like reaching out and gripping with the accuracy of 76% [13]. A possibility has been shown of discerning, with the use of EEG analysis, one of the five performed or intended movements of the wrist and the fingers with the accuracy of 65-71% [14]. At the same time, contrary to the established opinion about the necessity of using many EEG channels for accurate diagnosis, Yang J. et al. (2012), by discarding noise, picked 6 out of 32 channels, and introduced an updated classification of EEG signals for an artificial neural network and to control a robot. Classification accuracy in performing certain motor tasks was achieved at the level of 86% [11].

The most commonly used method of recording EEG signals for study brain potentials for both clinical and non-clinical purposes is the international standardized 10/20 system. Digits 10 and 20 stand for the distance between the electrodes in percent of skull circumference. The sampling frequency for picking the EEG signal is typically set below 200 Hz [15]. According to the studies of A. Kaplan, two main approaches are used for EEG signal segmentation: a) segmentation with a fixed interval that divides the EEG signal into segments of fixed length, b) adaptive segmentation, where the EEG signal is separated into quasi-stationary segments of varying length. The methods of adaptive segmentation may, in their turn, be divided into parametric, which make it possible to adequately describe the fragmentary stationary structure of the EEG signal, and non-parametric methods, where numerous or stringent assumptions about the properties of the EEG signal are not used, and predefined information about the distribution of random sequences is not required [16].

Other technologies of analyzing brain activity include magnetoencephalography (MEG) that measures magnetic brain potentials, magnetic resonance imaging (MRI) and near-infrared spectroscopy (NIRS), which estimates blood oxygenation in the cortex during an activity [12]. Several studies were performed using these technologies, which have shown the potential of controlling a neuroprosthesis. For example, Sugata H. et al. (2012) classified MEG signals for three types of right upper limb movements (grip, pinch and elbow flexion) with the average accuracy of 66 ± 10% [17]. Quandt F. et al. (2012) in course of parallel MEG and EEG established a higher accuracy of decoding the MEG signal, as compared to EEG, in the task of diagnosing the finger movements in pressing a single button with different fingers [18]. Lee J. H. et al. (2009) performed a pilot study of assessing the ability to control a robotic hand using only the mental processes of the person under test with the help of BMI in course of a real-time functional MRI [19]. They found that although MEG and fMRI are able to obtain the patterns suitable for controlling external devices, the used equipment is bulky and non-mobile. In addition, MEG devices are susceptible to magnetic noise from the body and urban environment, when operated outside magnetically isolated areas [12, 20]. Thus, MEG and MRI are not suitable for controlling limbs neuroprostheses and exoskeletons, unless there is a breakthrough in technologies of registering signals along with miniaturization.

On the contrary, NIRS-based BMI allows getting signals from moving objects [21]. NIR spectroscopy may be used not only as a BMI, but for identifying muscle activity as well. In case of muscular contraction, the features of the spectroscopy signal in the near infrared range show variation in the amplitude caused by blood outflow from the muscle [22]. However, there is no research so far that would confirm that a single-mode NIR spectroscopy may be used for discerning arm movements.

Problems and lines of development

With the development of sensor technology, many interfaces have been proposed for controlling robotized prostheses, however, stability of bio-signals is still far from perfect. Some difficulties in creating a stable control system should be mentioned. First, in the biosensor technologies, the common problem is electrodes misalignment, whereby intensity, quality and repeatability of bio-signals depend on correct electrodes positioning. In this regard, approaches to electrode placement that would prevent misalignment should be developed. Second, bio-signals are highly susceptible to distortion by various noises. Third, there is evident signal distortion when the user performs a complex movement, for example, moves fingers for gripping while changing the position of the entire limb. Solution of this problem requires cutting-edge technologies that would make it possible to discern the intended movements from the interference movements. Fourth, invasive interfaces to the peripheral nervous system and ECoG are susceptible to degradation of signal quality in case of long-term use, which results in the need for further research to improve stability of the system with neural interfaces of this type [23].

Against the background of the difficulties associated with ensuring reliable control using single-mode sensors, fusion of signals from multiple sources looks promising. Single-mode sensors are susceptible to insufficient data volume and specific noise contamination, and the fusion technology offers a complementary approach. For example, EMG signals are susceptible to influence as a result of muscular fatigue, and NIR spectroscopy can diagnose it, which provides the possibility to integrate an EMG-based NIR spectroscopy system sensor for detecting fatigue and compensating restrictions of pure EMG. The line of development of sensor data multi-mode merging is the integration of the technology into hardware and software. As there miniature sensors and embedded systems with significant processing power are currently available, such integration seems feasible.

Two approaches may be used for merging data from multiple sensors. The first is called “two-stage approach”, where the position of the limb is detected by a single-mode accelerometer, and the motion is discerned based on the data about its position. When the second approach is used, the length of data characteristics vectors is simply increased by adding the characteristics of the accelerometer signal, or other signal source.

Based on the foregoing, one can draw the conclusion that BMI-based robot control systems are becoming increasingly popular. From all methods of recording bio-signals in such systems, EMG, EEG and EOG are most commonly used, being the ones that meet the most the requirements of mobility, resistance to external interference, ease of maintenance and possibility to work in real time. Despite the significant progress in this area, no systems exist yet that would fully ensure intuitive and natural control. The main problem is relatively low accuracy of recognition for sufficiently complex systems with many degrees of freedom. The most promising approach to solve this problem is combining data from multiple bio-signals sources in order to improve the efficiency of classification. Such an approach, along with competent selection of detected characteristics and methods of classification will make it possible to improve accuracy and reproducibility in real conditions and, consequently, to improve interaction between man and machine. Thanks to the technological progress, such systems will soon become simpler and more intuitive, and will not require special training to be used; they will be able to recognize intentions and emotions of a person and adapt themselves to the dynamic changes of the environment.

In this regard on the basis of the neurobiology and medical physics laboratory of the Chemistry and Biology Institute of BFU n.a. I. Kant, research has been performed with the purpose of creating a multifunctional neuro-device and studying the possibilities of long-term monitoring of EEG, EMG, EOG, photoplethysmography (PPG), tissues saturation with oxygen (SpO2), body temperature, and motor activity with data transmission to exoskeleton devices.

In this regard, in course of implementation of this project one should solve two main tasks. (1) To develop a hardware solution for picking relevant bio-signals and auxiliary PPG signals, temperature and acceleration using EEG, EMG and EOG, to provide sufficient amount of data for stable operation of the discernment algorithms and for predicting physical activity. Design of a device that would combine picking several types of electrical bio-signals will make it possible to compensate for possible mutual interference from various types of signals, and to optimally combine picking methods in a single device. (2) To develop the algorithms of working with several data sources for improving the accuracy of discernment and flexibility of device operation. Such algorithms would make it possible to effectively manage exoskeletons and other robots in the conditions of natural ambient noise, as well as in cases when the level of user’s bio-signals in some channels is extremely small or missing due to physiological features.

The main problem of the research is real-time analyzing and predicting motor activity, based on user’s bio-signals picked by the device, and generating corresponding control commands for the exoskeleton. Solution to this problem may be divided into two components: hardware and methodical. In other words, firstly it is important to obtain the whole set of main bio-signals using EMG, EEG, EOG, and other signals, and to develop the software that would perform complex analysis of incoming data and generate real-time control commands for the exoskeleton.

This paper presents intermediate results of the research aimed mainly at developing and manufacturing a mock-up of the device for discerning electrophysiological signals and for testing.

Methods

This research was performed Stagewise.

In the first stage, world literature was analyzed, patents were elaborated and market was studied, and on the basis of the obtained data, the most promising and optimal lines of development have been identified. Moreover, components for creating the layout of an intellectual neuro-device have been selected, which have become a key object of the research and creation of a neuro-device that is capable to simultaneously discerning various electrophysiological signals (EEG, EMG, EOG with connecting photoplethysmogram modules, SpO2 and temperature), for ensuring the technology of full biological feedback, and for transferring processed information to the exoskeleton and the robot robotic in real-time.

In the second stage, the layout of the neuro-device was made.

In the third stage, the neuro-device was tested. The EEG method was used to measure brain electrical activity picked from the scalp. Using the electromyography method, bio-electrical potentials that occur in the skeletal muscles were measured. Using the method of EOG, potentials were measured during eyeball movement, wherein electrodes were installed around the eyes. Using the method of thermometry, temperature of the person under the test was determined. Using the method of physical exercise, motor activity was studied by the changes in the Euler angles. Using the method of photoplethysmography, heart rate was determined. All obtained data were recorded graphically and digitally. Figure 1 shows the research methodology for each method.

Figure 1: Methodology of testing: 1) motor activity; 2) EEG; 3) EOG; 4) PPG and SpO2; 5) EMG, 6) thermometry. Click here to View figure

In course of studies for assessing characteristics of physiological signals, most commonly encountered artifacts were determined: artifacts from poorly fastened electrodes, electrical interference caused by the motion of the person under test, artifacts caused by muscle tension in the body and wrinkling the forehead, muscle potentials, skin potentials, winking, and pulse waves.

Reference devices that have been well established in medical and neurophysiological market were used for the purpose of performing objective efficiency analysis of the developed device. Thus, the results of recording EEG, EMG and EOG signals were compared to the results of the KARDi3 device (Medical Computer Systems, Russia) intended for recording and analyzing ECG, EOG, EEG, respiration by chest excursion, abdominal wall and nasal-oral air flow, galvanic skin response from the object’s fingers and toes by means of an accelerometer or a sensor under the chair, changes in the volume of arms and legs muscles with the use of a sphygmogram and rheogram. This device is positioned as a polygraph detector, and is widely used for instrumental lie detection. For the purpose of comparing accuracy of body temperature measurement, the Fluke 17b multimeter with a plug-in thermistor (Fluke Corporation, USA) was used.

Discussion and Results

In course of performing the work, the optimum path and layout of the neuro-device were selected, software was written, and the experiments were performed. Let us present the steps of each stage.

Selection and justification of the optimum way to solve the problem of creating the neuro-device

Determination of optimum electrodes and materials.

The mechanism of electrical conductivity in a living organism is based on ions, which  carry the charge. The electrodes serve as mediators between the body and the analogue interface in course of recording bio-electric potentials. There are two types of electrodes: ideally polarizable and non-ideally polarizable. In the former, charges do not penetrate the interface of the electrode and the electrolyte, therefore they act as capacitors, and the electric current in them is determined by bias currents. In the latter the electrons, on the contrary, freely pass through the interface without changing the charge distribution in the electrolyte solution close to the electrode, so they behave as resistors. If an electrode is moved relative to the electrolytic solution, charge distribution in the solution near the surface changes, which may, in turn, cause a change in electrode voltage, and consequently create a motion artifact during measurement [24]. For medical measurements, non-polarizable electrodes are preferred to the polarizable ones. Table 1 shows most optimal types of electrodes for picking signals of heart, muscle and brain activity.

Table 1: The choice of electrodes for various bio-electric potentials

In manufacturing the device, modern and safe materials and coatings will be used [25, 26].

Determining the best power source and its operation mode.

For many years, Nickel-Cadmium batteries have been the only suitable variant for mobile devices, but with the advent of lithium-based rechargeable batteries, the situation has changed, and today they may be found everywhere.

Despite the advantages, lithium batteries have disadvantages: the device is fragile and requires a robust case for safe operation. In addition, there is temperature limit. Another problem is degradation over time, which is usually noticeable after a year of active use, while the total lifetime of a battery does not exceed 2 to 3 years.

Power consumption is one of the most important parameters in the development of any autonomous device. Reducing it makes it possible to increase autonomous work duration and reduce the battery cost. The electrical circuit includes a micro-controller that consumes current itself, and a specific liner, current consumption may also be substantial.

The most common modes are the Power Down (PWD) mode, the power save (PS) mode, and the idle mode. In the power saving mode, the CPU is not working, except for the clock frequency of the clock crystal. In the idle mode, a part of the CPU is disabled, but the basic functions are still being performed. The advantage of having several modes is that operation flexibility is ensured for disabling the functions that are not absolutely necessary. For example, the AVR ATmega 165 micro-controller with power supply voltage of 1.8 V, operating at 1 MHz, consumes 350 µa in the active mode, 150 µa in the idle mode, 0.65 µa in the power save mode, and 0.1 µa in the Power Down mode.

Control of the entire system is performed via micro-controllers, each with a unique set of commands and register; therefore a program written for one will not work for another one from another manufacturer. Table 2 shows main manufacturers and available micro-controllers’ models.

Table 2: Micro-controllers’ manufacturers and basic models available

The data transfer module is also an integral of the entire system. It is required for communication between devices via a wireless channel. It is typically implemented using channels such as ZigBee, Bluetooth and Wi-Fi. The information of current consumption for each of these types is shown in Table 3.

Table 3: Characteristics of wireless communication channels

Defining the best amplifiers of useful signal and the system of filtering external noise

Electric bio-electric potentials are low amplitude and low frequency signals. In addition, measuring of the currents is influenced by biological and ambient interference; therefore, it is clear that even in the most modern systems for measuring bio-electric potentials it is the analog interface that should meet the most stringent requirements. Its main task is amplifying and filtering weak bio-electric signals. However, such an amplifier should cope with various problems arising in the process of extracting the bio-electric signal, and its power consumption should be optimized and ensure long autonomous operation.

Low-frequency signals with the amplitude of about several microvolts in the frequency range of interest are dominated by flicker noise. Therefore, a low noise analog interface is required for picking high-quality signal.

Capacitive coupling with AC cables (50/60 Hz) generates an inphase signal at the input of the entire system, which requires ultra-high damping coefficient of the inphase signal. It is necessary to consider the electrode bias voltage generated at the point of contact with the skin. For this purpose, a high-pass filter should be used. There should be an option of configuring the analog interface for working with signals in various frequency ranges and having various amplitudes.

The balance between signal quality and power consumption is essential.

The most important component in the scheme of the analog interface is the first stage of the measuring amplifier (MA). It determines signal quality, noise level, common-mode rejection ratio (CMRR), and filters out the bias voltage generated by the electrodes. That is why design and optimization of this component should be paid special attention to, since each bio-electric potential has several distinct characteristics, which fact results in differences in the design of each amplifier. These differences are shown in Table 4.

Table 4: Electrical characteristics of various human bio-electric potentials

In measuring bio-electric potentials, common-mode interference makes a significant contribution to signal distortion.

In modeling, body capacity relative to earthing (Cbody) is usually is 300 pF, and the body capacity relative to power cables (Cpow) is equal to 3 pF. These capacities are the cause of the interference current with twice the amplitude of 0.5 µa flowing through the body from the source (220 V, 50 Hz) to earth. Very often the difference between Cpow and Cbody is so great that the interference current exceeds 0.5 mA ten times. If an amplifier is connected to the body, a part of this current will go to earth via resistor (Zr1) that corresponds to full impedance at the point of electrode and skin contact. As a consequence, the average difference of potentials between the body and the signal earthing of the amplifier will be non-zero, i.e. it will have common-mode voltage.

Due to the presence of interference currents, the amplifier should have very high CMRR (90 dB). The voltage divider effect is the main reason why it is important to reduce the common mode voltage component as much as possible.

The amplifier input should have sufficiently high impedance and should be well correlated with the signal source.The amplifier input should have sufficiently high impedance and should be well correlated with the signal source.
Capacitive relation between connecting wires and power cables is also a source of interference in bio-electric potentials measurement. Since the currents induced in the wires and in full electrodes resistors, as a rule, vary greatly, sufficiently large differential voltage occurs between the inputs of the amplifier.
The magnetic field generated by alternating current in power lines passes the closed loop formed by the body, the wires and the amplifier. It generates the electromotive force (EMF) in the circuit, to generate in turn AC voltage at the input of the circuit. Such interference is easily discerned from other types of interference, since it depends on the area and orientation of the closed circuit. Theoretically, one can get rid of them simply by plaiting wires and thus reducing the area of the circuit. However, this method is not always practically feasible. For example, in case of standard electrodes layout for measuring ECG, two electrodes are placed near the soles, which inevitably results in increasing the distance between the wires. Shielding the patient with materials featuring high magnetic permeability is also usually impossible. In this case there are two options: either to place the patient away from all magnetic field sources, or to use portable systems for processing of bio-electrical signals located closer to the electrodes.
Thus, the signal is amplified by an instrumentation amplifier, and various filters are used for picking the signal.

Manufacturing the neuro-device mock-up

In light of the foregoing, one can draw the conclusion that EEG is performed by using the bowl-shaped electrodes made of Ag/AgCl or gold, placed on the capillary part of head with a cap or other attachment to ensure tight contact with the skin for minimizing the motion artifacts. For removing EMG, it is desirable to use a bowl-shaped or plate-shaped Ag/AgCl electrodes fixed to the target muscle using a cuff, the use of gold electrodes is impractical due to their high prices. EOG is performed with the use of Ag/AgCl electrodes with bowl-shaped or plate-shaped electrodes fixed around eyes.

It is preferable to use an instrumental amplifier with feedback current based on a single chip with a system of interference filtration.

Autonomous power supply should be from a lithium battery. An intelligent power control system should be available for switching to standby mode with reduced functionality and power consumption.

The best choice for wireless data transmission is Bluetooth, as it has the optimum transfer rate/current consumption ratio.

For that reason, in manufacturing a neuro-device mock-up for simultaneous discernment of various electrophysiological signals and transferring them to the exoskeleton and robot, the following was used: a voltage converter with charge pump without stabilization with output current up to 60 mA; a voltage regulator with low dropout voltage and output current of 250 mA; a low-noise 8-channel 24-bit analog interface; an 8-bit micro-controller with 16/32/64 KB in-system programmable flash memory; ultra-low noise linear voltage regulator with the output current of 250 mA for analog and RF circuits that do not require blocking capacitors; positive voltage stabilizers designed for currents up to 800 mA with low dropout voltage; 8-bit main transceiver with adjustable output voltage and a three-state output; 4-channel device for protection against ESD with ultra-low leakage current, linear low noise stabilizers of negative voltage with low voltage dropout, high rejection ratio of ripple voltage and output current of 200 mA; a field p-channel transistor; a ceramic resonator; a position sensor with nine degrees of freedom (three gyroscopes, three accelerometers, three magnetometers); Bluetooth; a micro-SD slot; film and ceramic capacitors; high-precision low-power digital temperature sensor with two-wire serial interface in the housing; and a module with proximity / ambient light sensors.

Based on these data, EEG was performed with bowl-shaped electrodes made of silver (Ag/AgCl) and affixed with a cap; for picking EMG and EOG – plate-shaped Ag/AgCl electrodes, while for picking EOG – gel Ag/AgCl electrodes.

The results of experimental work

This work includes the following studies of a mock-up of the developed device for discerning complex electro-physiological signals and transferring them to the exoskeleton (neuro-device) with plug-in photoplethysmogram and temperature modules.

The developed neuro-device has been tested. For that purpose, physical activity has been studied first. Then the neuro-device mock-up was comparatively assessed, and the results were recorded with the KARDi3 device in course of the following electro-physiological studies: EEG (100 tests), EMG (femoral muscle, 100 tests), EOG (100 tests). The heart rate was assessed via photoplethysmographic tests with the use of the neuro-device mock-up and via ECG module of the KARDi3 device (100 tests). At the end of the first phase of tests a study for determining the temperature was performed with the use of the neuro-device mock-up and the Fluke 17b device (136 tests).All tests were made on two healthy men (22 and 23 years old, 175 cm and 177 cm tall, with the weight of 70 kg and 75 kg).

In-house software was developed for recording EEG, EMG and EOG data from the neuro-device, and the “Neocortex” application was used for recording the data from the KARDi3 device. The same Neocortex application was used for studying the data from the KARDi3 in studying heart rate, and the HEART RATE MONITOR DEMO was used for the neuro-device mock-up.

The results of the mock-up tests for motor activity

Motor activity was studied by performing the exercise of head tilting and turning left, right, forward, backward, with the neuro-device mock-up affixed to it. To do so, an application was written for real-time visualization of Euler angles.

In course of studying motor activity in performing a series of exercises of head tilting and turning left, right, forward, backward, the results were obtained that indicate high accuracy and precision of the data obtained using the motion of the neuro-device mock-up. The chart shows changes in the Euler angles (X-axis – time, Y axis – angle): 1) Pitch – movement along the transverse axis (green); 2) Roll – movement along the longitudinal axis (blue); and 3) Yaw – movement along the vertical axis (red).

Some measurements featured gyro float associated with changes in the magnetic field from the battery. This is due to the relatively soft mounting of the battery to the mock-up, after the battery was rigidly fixed to the neuro-device, this aspect was eliminated. In case of insufficient fixation of the neuro-device mock-up on the body of the patient, simultaneous change of more than two angles was sometimes observed in course of performing the same exercise. This influence was eliminated by more rigid fixation of the neuro-device experimental sample. A simultaneous change of two angles was also observed in performing the same exercise caused by an error due to the multi-layered nature of electronics. The components of the gyroscope are placed on the movement module, the movement module is located on the neuro-device, and the neuro-device is mounted on the head of the person under test. In addition, it has been found that smooth movement of the neck substantially contributes to the results. In manufacturing the experimental sample of the neuro-device this fact has been taken into account, and the hardware and software auto-calibration of the position relative to the patient will be used. In the experimental sample, the basic parameters of physical activity will be linear acceleration of the accelerometer, angular acceleration of the gyroscope, and the magnetic field vector of the magnetometer.

The results of the mock-up tests using EEG

The main parameter recorded in this experiment was alpha rhythm. This is due to the fact that during the wakefulness phase, the alpha rhythm of a healthy adult is the most stable electrophysiological signal. The signal was recorded with the use of a circuit with two bipolar lead electrodes. The electrodes were affixed to the back of the head, and the reference electrodes were fixed on ear lobes. Wires from the electrodes were braided and combined into a harness to reduce artifacts. The person under test was constantly in the “reclined” position for the maximum relaxation of head and neck muscles and avoiding EMG artifacts. The electrodes on the body were static, and did not change their position in the process of switching from KARDi3 to the neuro-device mock-up.

The same modes were used for the neuro-device mock-up and the KARDi3 device: 30 Hz low frequency filter, 0.5 Hz high frequency filter, 50 Hz rejector filter, X-axis scan 30 mm/s, and the Y-axis scan 50 μV/mm.

The EEG data obtained with the use of the neuro-device mock-up showed artifacts similar to those in the EEG data obtained from the KARDi3 device, and the number of muscle and heart artifacts was comparable.

The results of the mock-up tests using EOG

The EOG signal was recorded with the use of a circuit with two bipolar lead electrodes. The electrodes were affixed to the temples, close to the right eye and on the forehead. First, measurement was performed with KARDi3, then with the neuro-device mock-up. The person under test was in the sitting position in front of a board with graphic symbols. The electrodes on the body were static, and did not change their position in the process of switching from KARDi3 to the neuro-device mock-up. The EOG signal was received during eye movement in performing the following visual exercises: center (yellow dot) – up (red circle) – center (yellow dot) – down (blue circle) – center (yellow dot) – left (red cross) – center (yellow dot) – right (blue cross) – center (yellow dot).

The same modes were used for the neuro-device mock-up and the KARDi3 device: 40 Hz low frequency filter, 1 Hz high frequency filter, 50 Hz rejector filter, X-axis scan 15 mm/s, the Y-axis scan 50 μV/mm.

The EOG data obtained with the use of the neuro-device mock-up showed at lease comparable informativeness, compared to those in the EOG data obtained from the KARDi3 device, and the number of artifacts was lower in the data obtained with the developed neuro-device mock-up This fact testifies to the reliability of obtained results, accuracy of the data and high quality of the neuro-device mock-up. The most common artifact was blinking that was manifested as sharp increase in the EOG signal amplitude, artifacts of facial muscles that occurred when the person under test was tired, were also registered.

In further stages of work, the motion artifacts of the EEG and EOG signals will be eliminated in the experimental sample with additional reject filter and software, and by improving the tooling for fixing the electrodes.

The results of the mock-up tests using EMG

The EMG signal was obtained in case of reducing femoral muscles by moving the right foot forward, imitating a step. Here the left foot remained motionless, and the right foot with the electrodes did not support the body. The subject was in the standing position, the left leg and the right arm were supports for the body, and the right leg with electrodes was relaxed. The distance between the electrodes was 5 cm. The signal was recorded with the use of a circuit with one bipolar lead electrode. The electrodes were affixed to the femoral muscle by means of adhesive rings. Wires from the electrodes were braided and combined into a harness to reduce artifacts.

The same modes were used for the neuro-device mock-up and the KARDi3 device: 100 Hz low frequency filter, 1 Hz high frequency filter, 50 Hz rejector filter, X-axis scan 120 mm/s, the Y-axis scan 10 μV/mm.

During the experiment it was found that the EMG data obtained with the neuro-device mock-up have the same artifacts as the EMG data obtained with the KARDi3 device. This fact testifies to the reliability of the obtained results, accuracy of the data, and high quality of the neuro-device mock-up. No artifacts were found in performing this exercise.

The results of testing the mock-up for detecting temperature, pulse and tissues saturation with oxygen

When pulse signal was picked, the subject was in the sitting position. To record the ECG signal, KARDi3 electrodes were affixed to the wrists by means of adhesive rings according to the scheme of placing a single bipolar lead electrode. Next, a finger of the subject was applied to the photoplethysmogram module of the neuro-device mock-up, and a signal determined by pulsation was picked at the output. Heart rate measurement with the neuro-device was supplemented by pulse oximetry (SpO2). The same modes were used for the KARDi3 device: 0.1 Hz low frequency filter, 50 Hz high frequency filter, 50 Hz rejector filter, X-axis scan 60 mm/s, and the Y-axis scan 20 μV/mm.

For thermometry, the temperature sensor was placed on the forehead of the subject by means of an adhesive ring. The temperature data was transmitted every second to the personal computer via Bluetooth.

The obtained results of the research for determining temperature indicate that the neuro-device mock-up temperature sensor features high stability and accuracy of measurement. The results of measurement were comparable with those of the reference instrument Fluke 17b.

The detected pulse signal was also obtained in two ways: by obtaining an electrocardiogram by means of KARDi3 and the photoplethysmogram module of the neuro-device mock-up. The data of R-R interval ECG assessment obtained with the use of the KARDi3 device was characterized by the same heart rate values as those measured by the photoplethysmography module of the developed neuro-device. The average (median) heart rate was 78 and 77 beats/min at KARDi3 and the neuro-device, respectively, in the first subject, and 72 and 71 beats/min, respectively, in the second subject. No artifacts affecting the result were detected.

It should be noted that the developed module for the assessing the cardiovascular system has an advantage, since it includes assessment of blood saturation with oxygen, which can also provide valuable information for controlling the exoskeleton.

Conclusion

It is obvious that it is promising to create a high-precision multi-function neuro-device that would make it possible to continuously compile physiological signals and transmit information to an external device (exoskeleton), which would provide the millions of the disabled with the possibility to restore active social and home life. The presented latest world’s achievements bespeak of the possibility of soon advent of a neuro-device for medical and social purposes in the market. Today, scientists from all continents are working on this issue. Given the urgency of this problem and high disability rate, such applied research and experimental development are performed in Russia as well. This publication presents the intermediate work results.

The data obtained in the course of this work allowed us to draw the conclusion about high quality of the developed neuro-device mock-up that consists of a portable wireless telemetry device for recording the electrophysiological parameters such as EEG, EMG, and electrooculograms, and the biometric parameters such as physical activity, surface temperature, and photoplethysmograms. The key novelty of this topic is simultaneous juxtaposition of several methods of picking electrophysiological parameters. For this purpose, specialized software has been developed and created, which is capable of transforming electrophysiological signals into hardware commands that are sent to the exoskeleton.

In the course of further work, the maximum possible leveling of artifacts, improvement of methods of signal filtering, software improvement and the miniaturization of the device are planned. Experiments will be performed simultaneously for hybridization several of physiological signals, which will significantly increase the accuracy of classification. Due to higher classification accuracy and flexibility of merging, a hybrid system will feature potentially greater reliability and performance.

It is expected that development of this approach with hybridization of many physiological signals will increase the accuracy of the commands sent to the robotic mechanical device, which will certainly create competitive advantages and open new opportunities for rehabilitation of people with motor dysfunctions.

Acknowledgments

This work was supported by the Ministry of Education and Science of the Russian Federation grant RFMEFI57815X0140 in Immanuel Kant Baltic Federal University. The authors would like to thank Danil Borchevkin, Alexey Belousov and Vladimir Savinov for preparing the manuscript.

References

1. Weeks, M., 2007. Digital Signal Processing Using MATLAB and Wavelets. Infinity Science Press, pp: 452.
2. Tonet, O., Marinelli, L. Citi, P. Rossini, G. Megali and P. Dario, 2008. Defining brain–machine interface applications by matching interface performance with device requirements. Journal of Neuroscience Methods, 167(1): 91-104.
3. Collinger, J.L., Wodlinger, J.E. Downey, W. Wang, E.C. Tyler-Kabara, D.J. Weber, et al., 2013. High-performance neuroprosthetic control by an individual with tetraplegia. Lancet., 381(9866): 557-564.
4. Buzsáki, G., C.A. Anastassiou and C. Koch, 2012. The origin of extracellular fields and currents—EEG, ECoG, LFP and spikes. Nature Reviews Neuroscience, Vol. 13, 6: 407-420.
5. Yanagisawa, T., Hirata, Y. Saitoh, H. Kishima, K. Matsushita, T. Goto, et al., 2012. Electrocorticographic control of a prosthetic arm in paralyzed patients. Annals of Neurology, Vol. 71, 3: 353-361.
6. Benz, H.L., Zhang, A. Bezerianos , S. Acharya, N.E. Crone, X. Zheng, et al., 2012. Connectivity analysis as a novel approach to motor decoding for prosthesis control. Neural Systems and Rehabilitation Engineering, IEEE Transactions on., 20(2): 143-152.
7. Chestek, C.A., V. Gilja, H. Blabe, B.L. Foster, K.V. Shenoy, J. Parvizi, et al., 2013. Hand posture classification using electrocorticography signals in the gamma band over human sensorimotor brain areas Journal of Neural Engineering, 10(2): 026002.
8. Fifer, M.S., S. Acharya, L. Benz, M. Mollazadeh, N.E. Crone and N.V. Thakor, 2012. Towards Electrocorticographic Control of a Dexterous Upper Limb Prosthesis. IEEE pulse, 3(1):38.
9. Hochberg, L.R., D. Bacher, B. Jarosiewicz, Y. Masse, J.D. Simeral, J. Vogel, et al., 2012. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature, 485(7398): 372-375.
10. Gu, Y., K. Dremstrup and D. Farina, 2009. Single-trial discrimination of type and speed of wrist movements from EEG recordings. Clinical Neurophysiology, 8(120): 1596-1600.
11. Yang, J., Singh, E. Hinesc, F. Schlagheckend, D.D. Iliescuc, M.S. Leesonc, et al., 2012. Channel selection and classification of electroencephalogram signals: An artificial neural network and genetic algorithm-based approach. Artificial Intelligence in Medicine, 55(2):117-126.
12. Yanagisawa, T., Hirata, Y. Saitoh, T. Goto , H. Kishima, R. Fukuma, et al., 2011. Real-time control of a prosthetic hand using human electrocorticography signals: technical note. Journal of Neurosurgery, 114(6):1715-1722.
13. Agashe, H. and L. Contreras-Vidal, 2011. Reconstructing hand kinematics during reach to grasp movements from electroencephalographic signals. Engineering in Medicine and Biology Society, EMBC, 2011. Annual International Conference of the IEEE, 2011: 5444-5447. DOI: 10,1109 / IEMBS.2011.6091389.
14. Mohamed, A.K., T. Marwala and L.R. John, 2011. Single-trial EEG discrimination between wrist and finger movement imagery and execution in a sensorimotor BCI. Engineering in Medicine and Biology Society. EMBC, Annual International Conference of the IEEE, 2011: 6289-6293.
15. Sörnmo, L. and P. Laguna, 2005. Bioelectrical signal processing in cardiac and neurological applications. Academic Press, pp: 688.
16. Iscan, Z., Z. Dokur and T. Demiralp, 2011. Classification of electroencephalogram signals with combined time and frequency features. Expert Systems with Applications, Vol. 38, 8: 10499-10505.
17. Sugata, H., T. Goto, M. Hirata, T. Yanagisawa, M. Shayne, K. Matsushita, et al., 2012. Neural decoding of unilateral upper limb movements using single trial MEG signals. Brain Research, 1468: 29-37.
18. Quandt, F., C. Reichert, H. Hinrichs, J. Heinze, R.T. Knight and J.W. Rieger, 2012. Single trial discrimination of individual finger movements on one hand: a combined MEG and EEG study. NeuroImage, 59(4): 3316-3324.
19. Lee, J.H., J. Ryu, A. Jolesz, Z.H. Cho and S.S. Yoo, 2009. Brain–machine interface via real-time fMRI: preliminary study on thought-controlled robotic arm. Neuroscience Letters, 450(1): 1-6.
20. Tonet, O.,M. Marinelli, L. Citi, M. Rossini, L. Rossini, G. Megali et al., 2008. Defining brain–machine interface applications by matching interface performance with device requirements. Journal of Neuroscience Methods, 167(1): 91-104.
21. Piper, S.K., A. Krueger, P. Koch, J. Mehnert, C. Habermehl, J. Steinbrink, et al., 2014. A wearable multi-channel fNIRS system for brain imaging in freely moving subjects. Neuroimage, 85: 64-71.
22. Herrmann, S., A. Attenberger and K. Buchenrieder, 2012. Prostheses control with combined near-infrared and myoelectric signals. Computer Aided Systems Theory. Springer Berlin Heidelberg, 6928: 601-608.
23. Judy, J.W., 2012. Neural interfaces for upper-limb prosthesis control: opportunities to improve long-term reliability. IEEE Pulse, 3(2): 57-60.
24. Feldman, Y., P.B. Ishai and V. Raicu, 2015. Electrode Polarization. Dielectric Relaxation in Biological Systems: Physical Principles, Methods, and Applications, pp: 140.
25. Bogdanov, E., Y. Vavilina, N. Shusharina, A. Goykhman and M. Patrushev, 2013. The particles precipitation and osseointegration of a Tio2 thin-film coating by ion beam deposition – An in vivo study. Journal of Nanomedicine and Nanotechnology, 5(1): art. 189.
26. Goikhman, A.Y., S.A. Sheludyakov and E.A. Bogdanov, 2011. Ion Beam Deposition for novel thin film materials and coatings. Materials Science Forum, 674: 195-200.