Transfer project strengthens robot autonomy and teamwork with humans

Autonomous mobile robots that work safe­ly and intu­itive­ly with humans are not only an impor­tant build­ing block of Indus­try 4.0. In future space mis­sions, they are expect­ed to sup­port infra­struc­ture con­struc­tion on for­eign plan­ets. In the now com­plet­ed trans­fer project Trans­FIT, the Ger­man Research Cen­ter for Arti­fi­cial Intel­li­gence (DFKI), the Uni­ver­si­ty of Bre­men and Siemens AG devel­oped the robot­ic capa­bil­i­ties nec­es­sary for the autonomous and col­lab­o­ra­tive exe­cu­tion of com­plex assem­bly work. The project was fund­ed through the Ger­man Aero­space Cen­ter e.V. (DLR) by the Ger­man Fed­er­al Min­istry of Eco­nom­ics and Cli­mate Pro­tec­tion (BMWK).

To pro­tect astro­nauts from adverse envi­ron­men­tal con­di­tions and to enable them to stay on alien celes­tial bod­ies for longer peri­ods of time, it is essen­tial to build up infra­struc­ture. Autonomous mobile robots are pre­des­tined for this demand­ing work, as they can per­form work that is poten­tial­ly dan­ger­ous or phys­i­cal­ly demand­ing for humans, either alone or as part of a robot­ic team. How­ev­er, the sys­tems reach their lim­its where human intel­lect, dex­ter­i­ty and cre­ativ­i­ty are required.

At the con­clu­sion of the Trans­FIT project launched in July 2017, the coop­er­a­tion part­ners from the Robot­ics Inno­va­tion Cen­ter of the DFKI, the Robot­ics Work­ing Group of the Uni­ver­si­ty of Bre­men and Siemens AG pre­sent­ed inno­v­a­tive tech­nolo­gies that not only strength­en the autonomous capa­bil­i­ties of the robots through arti­fi­cial intel­li­gence (AI) meth­ods, but also enable sit­u­a­tion­al­ly appro­pri­ate and intu­itive sup­port by humans. In addi­tion, the part­ners demon­strat­ed the high trans­fer poten­tial of the solu­tions devel­oped by set­ting up a flex­i­ble assem­bly cell for indus­tri­al pro­duc­tion. In addi­tion, a soft­ware frame­work devel­oped specif­i­cal­ly for autonomous pro­duc­tion, which plans process­es depend­ing on sys­tem capa­bil­i­ties, was incor­po­rat­ed into the space scenario.

“Future space mis­sions will increas­ing­ly rely on Arti­fi­cial Intel­li­gence meth­ods and the com­bi­na­tion of human and robot­ic capa­bil­i­ties. With the com­ple­tion of the Trans­FIT project, which is a sig­nif­i­cant part of the DFKI Space Roadmap, we have suc­ceed­ed in lay­ing impor­tant foun­da­tions for coop­er­a­tive infra­struc­ture devel­op­ment in space and demon­strat­ing the trans­fer­abil­i­ty of the devel­oped tech­nolo­gies into the indus­tri­al con­text. In per­spec­tive, both indus­tries will ben­e­fit great­ly from the use of autonomous robots that can work in teams with humans.”

- Prof. Dr. Dr. h.c. Frank Kirchner

Flexible human-robot collaboration for infrastructure construction in space

In the extrater­res­tri­al con­struc­tion of sta­tion­ary camps, shel­ters or facil­i­ties for resource extrac­tion, robots have to cope with tasks of vary­ing com­plex­i­ty. This also requires col­lab­o­ra­tion with astro­nauts. Trans­FIT there­fore focused on the imple­men­ta­tion of a coop­er­a­tion sce­nario in which a human and a humanoid robot joint­ly per­form an assem­bly task. They are sup­port­ed by a per­son who can con­trol the robot from a greater dis­tance, for exam­ple from the sta­tion in orbit or from the lan­der, with the aid of an exoskeleton.

The coop­er­a­tion part­ners inter­act to vary­ing degrees accord­ing to the con­cept of “slid­ing auton­o­my”, from com­plete auton­o­my to auton­o­my with “oper­a­tor in the loop” and tele­op­er­a­tion with par­tial­ly autonomous func­tions to pure tele­op­er­a­tion. To real­ize this switch between autonomous work, coop­er­a­tion and tele­op­er­a­tion, the project part­ners devel­oped easy-to-use con­trol soft­ware that allows rapid adapt­abil­i­ty of the robot­’s behav­ior on site and dur­ing the mis­sion. In this way, even unfore­seen assem­bly tasks such as unsched­uled repairs can be per­formed flex­i­bly by the robot in coop­er­a­tion with the astro­naut instead of autonomously.

Learning assistant robot for complex assembly work and teleoperation

As a robot­ic test plat­form, DFKI researchers built the humanoid RH5 Manus, an assis­tance robot devel­oped for use in a human envi­ron­ment and fea­tur­ing a hybrid ser­i­al-par­al­lel design. Thanks to its inno­v­a­tive adap­tive grasp­ing sys­tem, which fea­tures tac­tile sen­sors, near-field recog­ni­tion and a local micro­con­troller for bio­log­i­cal­ly inspired grasp­ing reflex­es, the robot is capa­ble of per­form­ing com­plex grasp­ing move­ments. Machine learn­ing-based envi­ron­men­tal per­cep­tion meth­ods such as object recog­ni­tion and grasp pose deter­mi­na­tion enable it not only to grasp and hold com­po­nents, but also to coop­er­ate direct­ly with the astronaut.

In addi­tion, RH5 can be remote­ly con­trolled with the help of an active exoskele­ton devel­oped at the Robot­ics Inno­va­tion Cen­ter and opti­mized for tele­op­er­a­tion. How­ev­er, the goal of the col­lab­o­ra­tion is not only the divi­sion of tasks, but also that the robot learns from the inter­ac­tion with humans in order to act more and more autonomous­ly and to be able to adapt to chang­ing require­ments. In extrater­res­tri­al mis­sions, for exam­ple, it must be borne in mind that the sys­tem can­not be repaired at all or at any time. If, for exam­ple, the robot is restrict­ed in its move­ment because one of its joints no longer func­tions prop­er­ly, it can adapt its behav­ior to take this restric­tion into account thanks to the tech­nolo­gies devel­oped in TransFIT.

Improved teamwork thanks to intuitive interaction options

For the most intu­itive col­lab­o­ra­tion pos­si­ble between humans and robots, the inter­ac­tion part­ners are able to com­mu­ni­cate via speech using pre­de­fined com­mands. In addi­tion, the robot can be con­trolled by sim­ple ges­tures that are rec­og­nized with the help of a sen­sor suit worn by the astro­naut or the astronaut.

The project part­ners are also using the human elec­troen­cephalo­gram (EEG) to improve the inter­ac­tion: First, they devel­oped meth­ods to deter­mine the cog­ni­tive load of the inter­act­ing per­son from the mea­sure­ment data, which can be used to pro­vide sup­port as need­ed. On the oth­er hand, in the event of an erro­neous action by the robot, a neg­a­tive feed­back sig­nal can be obtained from the EEG, the so-called error poten­tial. This enables the sys­tem to con­tin­u­ous­ly improve its behav­ior and relieves the human, who does not have to con­scious­ly pro­vide the feed­back to the robot.

Transfer of space technologies to the context of Industry 4.0

In addi­tion to imple­ment­ing the extrater­res­tri­al coop­er­a­tion sce­nario, Trans­FIT aimed to trans­fer the devel­oped tech­nolo­gies to ter­res­tri­al appli­ca­tions, specif­i­cal­ly to indus­tri­al man­u­fac­tur­ing and pro­duc­tion. Here, the increas­ing num­ber of vari­ants of the prod­ucts to be man­u­fac­tured and the ever short­er prod­uct life cycles are lead­ing to ever small­er batch sizes. A cir­cum­stance that makes the use of clas­sic automa­tion par­a­digms almost impos­si­ble, even in assem­bly, which is large­ly depen­dent on man­u­al work. Against this back­ground, the project part­ners, under the lead­er­ship of Siemens AG, devel­oped a high­ly flex­i­ble and coop­er­a­tive assem­bly cell for the pro­duc­tion of com­plex assemblies.

The cell, which has two robot­ic arms, is capa­ble of imple­ment­ing abstract task spec­i­fi­ca­tions autonomous­ly and in col­lab­o­ra­tion with a human work­er. To achieve this, object recog­ni­tion, grasp pose deter­mi­na­tion and intu­itive human-machine inter­ac­tion tech­nolo­gies were trans­ferred from the space sce­nario to the indus­tri­al con­text. In addi­tion, a skill-based frame­work devel­oped specif­i­cal­ly for autonomous man­u­fac­tur­ing enables hier­ar­chi­cal mod­el­ing and orches­tra­tion of sequences in the man­u­fac­tur­ing process based on seman­tic prod­uct descrip­tions and required sys­tem capa­bil­i­ties. This frame­work has in turn been suc­cess­ful­ly trans­ferred to human-robot col­lab­o­ra­tion in space to assign the actions required for a task to the actors and to plan the sequence of actions.