interact interactive manual assembly ... - european commission · the interact project (611007) is...
TRANSCRIPT
The INTERACT project (611007) is co-funded by the European Commission under the 7th Framework Programme.
This document reflects only authors’ views. The Europena Commission is not liable for any use that may be done of the information contained
therein.
INTERACT – Interactive Manual Assembly Operations for
the Human-Centered Workplaces of the Future
Grant Agreement Number : 611007
: INTERACT
Project Start Date : 1st October, 2013
Consortium : DAIMLER AG (DAIMLER)- Project Coordinator
ELECTROLUX ITALIA S.P.A. (ELECTROLUX)
INTRASOFT INTERNATIONAL SA (INTRASOFT)
IMK AUTOMOTIVE GMBH (IMK)
EMPHASIS TELEMATICS AE (EMPHASIS)
HADATAP SP ZOO (HADATAP)
UNIVERSITY OF PATRAS (LMS)
UNIVERSITAET ULM (IMI)
DEUTSCHES FORSCHUNGSZENTRUM FUER KUENSTLICHE
INTELLIGENZ GMBH (DFKI)
Title : Requirements on efficient manual assembly model generation and interaction
Reference : D1.1.1
Availability : Public (PU)
Date : 31/03/2014
Author/s : LMS, DAIMLER, ELECTROLUX, DFKI, IMK
Circulation : EU, INTERACT consortium
Summary:
Description of the INTERACT requirements regarding the generation, manipulation and improvement
of digital human simulation models used for planning human work tasks. This deliverable is the main
outcome of Task 1.1.
INTERACT 611007
1
Contents
1. EXECUTIVE SUMMARY ..................................................................................................... 2
2. INTRODUCTION ................................................................................................................... 3
3. CURRENT PRACTICES AND STATE OF THE ART IN MANUAL PROCESSES
MODEL GENERATION AND INTERACTION .................................................................. 4
3.1. Current practices .............................................................................................................. 4
3.2. State of the art .................................................................................................................. 7
3.2.1. Motion generation and motion synthesis ............................................................ 7
3.2.2. Ergonomics Assessment ..................................................................................... 8
3.2.3. Controlled Natural Language ............................................................................. 9
4. BASELINE AND RESULTS ASSESSMENT ..................................................................... 11
4.1. EMA ............................................................................................................................... 11
4.1.1. Introduction ...................................................................................................... 11
4.1.2. Creating a scene ................................................................................................ 11
4.1.3. Task/movement definition ................................................................................ 12
4.1.4. Assessment tools .............................................................................................. 13
4.2. Parameters & Methods for software evaluation ............................................................. 14
4.3. Success criteria ............................................................................................................... 15
4.4. Assessing the adequacy of 3D human simulation models for process verification in
the industry ................................................................................................................. 17
5. REQUIREMENTS ANALYSIS ........................................................................................... 19
5.1. Introduction .................................................................................................................... 19
5.2. Vision and Key Features ................................................................................................ 19
5.2.1. 3D DHM level .................................................................................................. 19
5.2.2. Design engineer/process planner expert level .................................................. 20
5.2.3. Ergonomics expert level ................................................................................... 20
5.2.4. Software integration ......................................................................................... 20
5.3. DHM motion generation and interaction system requirements ...................................... 20
5.3.1. Elementary motions to be simulated ................................................................ 20
5.3.2. Derivation of a sequence of elementary motions from planning texts ............. 22
5.3.3. Motion constraints generation .......................................................................... 24
5.3.4. Motion synthesis ............................................................................................... 25
5.3.5. Model interaction.............................................................................................. 26
5.3.6. Motion simulation / animation ......................................................................... 27
5.3.7. Ergonomics assessment .................................................................................... 28
5.3.8. Non-functional requirements ............................................................................ 28
6. CONCLUSIONS ................................................................................................................... 29
7. GLOSSARY .......................................................................................................................... 30
8. REFERENCES ...................................................................................................................... 31
9. APPENDIX I: USER REQUIREMENTS CATALOGUE FOR DHM BASED MODEL
SIMULATION ...................................................................................................................... 34
INTERACT 611007
2
1. EXECUTIVE SUMMARY
The content of this document is the outcome of INTERACT Task 1.1 “Requirements on efficient
manual assembly model generation and interaction”. The main purpose of this document is to:
Identify important problems in current practice in terms of Digital Human Model (DHM) based
model generation and modification
Define an approach for measuring the success of INTERACT DHM motion simulation and also
provide a baseline to compare INTERACT achievements.
Define requirements for efficient model generation and modification
The document is structured as follows:
Chapter 3 provides an analysis of the current practices and state of the art. A list of key problems
identified in current industrial practice is reported. In the Appendix I, a detailed list of business
requirements is provided.
Chapter 4 provides a description of imk automotive EMA tool, which will provide the baseline for
comparing INTERACT achievements in the project. Moreover in chapter 4, a list of success
criteria and an approach to measure them are presented. Special focus is provided for measuring
the adequacy and the quality of DHM based simulation for digital validation. These criteria will
be used for comparing INTERACT achievements to the EMA (for most aspects) and DELMIA
V5 (for some aspects).
Chapter 5 presents key features and key system requirements for the INTERACT DHM based
simulation. The requirements focus on the following aspects:
o Elementary motions to be simulated
o Derivation of a sequence of elementary motions from planning texts
o Model constraint generation
o Motion synthesis
o Model interaction
o Model simulation / animation
o Non-functional requirements
Primary conclusions / results include the following:
The main result of this deliverable is the list of requirements and key features for the INTERACT
motion simulation developments to take place in WP2 “Best-fit simulation of manual assembly
using sensor data”. The system requirements and key features are derived by analyzing the
current practices and the INTERACT user requirements provided in the Annex.
The analysis of the state of the art has indicated that there is a high potential in using the
principles of the morphable graphs approach for a baseline in DHM motion generation.
The list of success criteria will be used in WP 6 “Industrial Pilot Cases, Demonstration and
System Validation” for assessing performance of the INTERACT motion generation and
synthesis approach.
INTERACT 611007
3
2. INTRODUCTION
The aim of the deliverable is to identify and report the requirements for generation of virtual models
that are used to represent and study manual operations including assembly and warehousing activities.
These requirements are gathered through the analysis of the current industrial practices in manual
assembly operations. The final target is to record areas for improvement in the current practices and to
set the basis and technological targets for the INTERACT modules that will be responsible for
building the human simulation models.
The capturing of the requirements will be mostly based on a) the analysis of existing tools/techniques
for model generation and the evaluation of their capabilities in terms of data input, efforts required
and time and b) on the analysis of existing practices that are considered by the end users when
creating such models for evaluating and optimizing their processes.
INTERACT 611007
4
3. CURRENT PRACTICES AND STATE OF THE ART IN MANUAL PROCESSES
MODEL GENERATION AND INTERACTION
3.1. Current practices
Production systems need to adjust to a continuously changing market. Trying to keep up with market
demands, OEMs such as automotive and professional white goods are offering an increasing number
of customized products and their variants. In order to address faster time-to-market requirements by
modern production systems are faced with faster and more ‘right-first-time’ ramp-up processes, which
can manage shorter production planning cycles and increasing product variety, as well as the need to
include new variants into existing production lines (Michalos et. al. 2010). Moreover, considerable
attendance has been given by the industry to minimize production cycle time, while considering
fatigue, safety and health issues of the human operators, especially in manual assembly processes
(Chryssolouris 2006). Therefore, it is necessary that a considerable amount of ergonomic factors be
analyzed and evaluated in order for injuries and safety problems to be avoided. The traditional
method of analyzing complex manual or hybrid production process is the implementation of a
product’s complete physical mockups, the assembly workspace and the real workers used, in different
combinations, in order for the optimum set-up to be produced. This is both time consuming, and
costly; characteristics that do not conform to the goals of modern industry. In automotive industry
prototype building serves as an opportunity to evaluate, verify and optimize production planning
(Manns and Arteaga 2013). The assembly planning team conducts several prototype builds before the
start of production. During these prototype builds, the planner reads the planned processes out loud,
and a worker interprets and follows the instructions, while experts from different fields perform
different evaluations and verifications. The goal of these builds is to achieve an optimal production
process preparation through which efficient, ergonomically optimal, stable and robust processes are
defined, product quality is guaranteed, and production ramp-up can be completed faster. As product
variety increases, it becomes more difficult and costly to build physical prototypes of all variants.
However, reducing prototypes reduces the number of manual assembly process verifications that can
be done using prototypes (Manns and Arteaga 2013).
However nowadays, the use of the Digital Human Modeling (DHM) software integrated into the
Virtual Manufacturing Software (VMS) is common practice in many industries, especially those
producing complex products (e.g. vehicles and aircrafts) or working with complex production and
maintenance processes. A number of DHM tools have been made commercially available to support
human factors and the ergonomics analysis in a product, process and workplace design (e.g. Siemens
2014 and Dassault Systemes 2014). The DHM tools use computer manikins (i.e. digital humans) as
representations of the human operators inserted into a synthetic or virtual environment to facilitate the
assessment of process performance and safety. Apart from the human representation in a digital
format, the DHM software tools are usually equipped with a certain functionality that is suitable for
INTERACT 611007
5
considering and analyzing human factors and ergonomics aspects. The availability of accurate digital
models and simulation tools has significantly reduced development time and cost by being able to
identify and correct problems before any physical prototypes are built. For example in automotive
assembly planning digital buildability check is a well-established process, in which digital models of
the vehicle’s parts are checked for geometric consistency, thus guaranteeing that all parts can be
assembled before any prototype parts are manufactured.
Unfortunately, in practice, such digital assembly verifications mainly comprise static analyses of
buildability and accessability. This is due to time consuming modeling, inflexibility to changes on
process, and product, and to often occurring unnatural movements of the DHM if dynamic processes
are modeled. In order to successfully conduct DHM simulation in the design process, generating
human motion is one of the most important issues to be addressed. Forward and Inverse Kinematics
are used to generate motion, such as human walking, or a library of predefined postures may be
available for users. . In forward kinematics, instead of specifying positions, the motion animator
specifies parameters that condense the essence of the motion and allow its individualization. In
inverse kinematics, if the position of the final effector is given, the calculation of the intermediate
links is automatic. Typically, the user defines a number of key frames, i.e. static postures. Then
DHM interpolates between every pair of key frames to generate in between postures that depict the
whole movement of the DHM. Following this approach, this is the current practice, modeling a
simple task, such as walking and picking a part form a bin, requires the definition of various key
frames. However, if changes on the scene are required, e.g. a new position of the bin or a new version
of the part, new modeling of the whole process is usually required. Even after carefully modeling the
key frames, unnatural behavior of the DHM’s movements could arise. So as to avoid this issue,
usually more key frames need to be defined, thus increasing modeling time (Manns and Arteaga,
2013). In a similar manner ergonomic assessment is primarily provided as a static analysis of the task
at hand, i.e. one can create a static ‘‘snapshot’’ of the posture or situation to be evaluated and then
perform an analysis using the DHM tool (Alexopoulos et. al. 2013).
Another, technique for motion generation and simulation of manufacturing and assembly tasks is
Motion Capture (Hartel et. al. 2010). It is a technique to record and digitally present human body
movements. By attaching markers or sensors on the human body, cameras or receivers can capture the
motions of body segments. Comparing to the kinematics approach, generating motion through motion
capture system has the advantage of describing realistic human motions and requires less modelling
time. However, typical Motion Capture systems, such as optical and magnetic ones, are a) expensive
and b) are very difficult to setup and operate in an industrial working environment and c) their
generated motion neither be edited or adapted to different task constraints nor be compared to
simulated motions on the fly or offline in a time efficient manner. Consequently, Motion Capture is
INTERACT 611007
6
not effectively used for industrial process design although its benefits for generating realistic digital
simulations are widely recognized.
Alternatives, like imk automotive’s EMA, address the time consuming and inflexible approach of
classical DHM tools by allowing a more abstract modeling of the DHM movements and its
interactions (Manns and Arteaga 2013). This approach is based on the decomposition of tasks into
basic operations and the use of parameterized movement generation. The underlying movements
appear more natural due to the fact that these were synthesized by recording and analyzing the
movements of real workers executing diverse manual tasks (Fritzsche et al. 2011). However, as we
move away from detailed but time consuming DHM based process modelling then we lose a lot in
terms of simulation quality (for example in terms of simulation realism).
There are two main industrial pilot cases within the INTERACT project. The first is one is related to
digital automotive production planning verification, the second one to warehouse component handling
within the area of professional white goods. Both pilot cases are shortly described in D1.2.1
“Requirements on monitoring of manual assembly operations”. A complete description of the project
pilot cases is provided in confidential deliverable D1.4.1 “Industrial pilots definition”.
The analysis of the current practice in industry and the definition of the pilot cases has indicated the
key problems of using DHM based simulations in current practice. Existing simulation tools have
proven to be difficult to cope with, especially with respect to the aspects summarized in the following
Table 1.
Key Problems Identified in Current Industrial Practice Importance
Too high modeling time required by the user. Critical
Displayed process could not be changed on the fly. Instead editing of the model
requires even up to 1-2 weeks.
Critical
Unlike with the evaluation using physical parts, during the use of virtual models
there are less solution suggestions.
High
Ability to modify the digital model using sensor data from some motion capture
session.
High
High realism of the model that satisfies needs especially of ergonomics experts.
There should be minimum trade-off between modelling time and realism.
High
Forces and their impact on workers were not displayed High
It is not possible to compare 3D human simulation process models with real Average
INTERACT 611007
7
human process execution both on the fly and off-line (in a time efficient manner
for the later).
3D human simulation models should be interactive and flexible so that they can
be altered by different process/product designers by him/herself without the need
of an expert with DHM modelling skills.
Average
The users of the models do not have the ability touch parts and get a real feeling
of the requirements for handling/assembling the part
Average
Table 1: Key problems for efficient 3D human simulation models generation and interaction
Following the identification of the key problems for using DHM based simulations for analyzing the
industrial processes a detailed list of requirements has been generated as an outcome of the
requirements gathering phase. The comprehensive list is provided in Appendix I: USER
REQUIREMENTS CATALOGUE FOR DHM BASED MODEL
3.2. State of the art
3.2.1. Motion generation and motion synthesis
Manual assembly stations incorporate complex and time-consuming operations that involve human
interaction with different kinds of tools and materials. DHM tools have been developed to
characterize human locomotion and to simulate human-system interaction (Mavrikios et. al. 2007,
Pappas et. al. 2007). In order to successfully conduct DHM simulation in the design process,
generating human motion is one of the most important issues to be addressed. Forward and Inverse
Kinematics are used to generate motion, such as human walking. These systems are based on
biomechanical and biological studies. In forward kinematics, instead of specifying positions, the
motion animator specifies parameters that condense the essence of the motion and allow its
individualization. In inverse kinematics, if the position of the final effector is given, the calculation of
the intermediate links is automatic (Chryssolouris et. al. 2001). Other approaches for motion
simulation utilize a database of motions for motion modelling and prediction (Park et al. 2004) or
combine existing motions in order to generate new ones. Several approaches use statistical analysis
(e.g. regression) on motion-captured data in order to form predictive models for a sequence of
postures. Motion adaptation/retargeting techniques, influenced by computer graphics ones have been
used in order to generate new motions for different characters with the existing captured motions.
These approaches consider motion as a set of signals and apply signal-processing techniques to them
(Bruderlin and Williams 1995, Witkin and Popovic 1995). Retargeting techniques usually re-use
motions created for one character for controlling the motion of others, so as to preserve as many as
possible of the desirable properties of the original motion. In that case, constraint optimization
INTERACT 611007
8
techniques are used to impose valid postures (Gleicher 1998) or multiple-objective optimization has
been proposed in (Yang et. al. 2004). One popular data-driven approach is motion graphs, which
represent allowable transitions between poses (Kovar et al. 2002; Safonova and Hodgins 2007).
Motion graphs create an animation by cutting pieces from a motion database and reassembling them
to form a new motion.
In (Heck and Gleicher 2007) they present the parametric motion graph, an example-based motion
synthesis data structure. A parametric motion graph describes possible ways to generate seamless
streams of motion by concatenating short motion clips generated through blending-based parametric
synthesis. Blending-based parametric synthesis allows generation of any motion from an entire space
of motions, by blending together examples from that space.
In order to overcome drawbacks of the motion graphs concept, (Min et. al. 2012) have developed a
generative motion model that is compact and amenable for motion analysis and synthesis. Elementary
motions such as right stance or left stance are represented by parameterized statistical distributions.
These distributions form the nodes of a graph (called morphable graph or motion graph++), in which
transition probabilities of the parameters are represented by the vertices, i.e. if in a walk the right
stance is done slowly, the following left stance is likely to be slow, too (Figure 1). This representation
allows describing infinite elementary motion styles within one node and therefore allows efficient
storage and synthesis with huge and heterogeneous motion databases. These styles are differentiated
from high-level structures, that are derived from traversing different graph paths. Min et. al. have
demonstrated that their system can generate a wide range of style variations, including functional
variations (e.g. locomotion speeds, step sizes, uneven terrains, and turning angles).
Figure 1: A morphable graphs model for running (Min et. al. 2012)
3.2.2. Ergonomics Assessment
Scientific results show that computer manikins are viable tools for the verification of ergonomics
early in a development process and for helping detect many problems prior to physical pre-series.
There is a considerable amount of research that focuses on the application of the DHM technology,
INTERACT 611007
9
incorporated in VMS for the evaluation of ergonomics in industry by using commonly available
methods. Demirel and Duffy (2007) have investigated the use of DHM integrated within PLM for
three industrial cases and their results indicated that the DHM tools have the potential to improve
design challenges during product development and provide control of the entire process of designing
and analyzing a product before ever being launched. Mavrikios et al. (2007) and Pappas et al. (2007)
have applied static posture analysis, in a real-life scenario, by using ergonomic tools provided by
DHM and they proposed countermeasures in order to improve working “comfort” in manual assembly
tasks.
Dynamic ergonomics analysis in a VMS was presented in Rhen et. al. (2011a) and Rhen et. al.
(2011b). It is illustrated how time significant wrist exposure data can be obtained from a DHM and
may be used for ergonomics assessment. An assembly task is analysed with the help of DHM tool
and they propose an evaluation model where RULA’s thresholds are used in combination with time-
dependant information regarding wrist flexion/extension. Ma et. al. (2010) propose a framework to
evaluate manual work operations with the support of motion tracking, DHM and the Maynard
Operation Sequence Technique (MOST) motion–time method. Their framework utilizes a
mechanism of automatic motion recognition but their work focuses on assessing manual work
efficiency, using MOST, rather than on assessing ergonomics related risks of manual operations.
Alexopoulos et. al. (2013) presented ErgoToolkit, which implements ergonomic analysis methods,
into digital tools for ergonomics, integrated into state of-the-art virtual manufacturing software
DELMIA V5. Their approach implements the Automotive Assembly Worksheet (AAWS) method
presented in Winter et al. (2006).
3.2.3. Controlled Natural Language
Controlled Natural Languages (CNLs) are subsets of natural languages such as English. Not all words
are permitted, not all grammatical constructions are allowed, and certain stylistic rules are in place to
guide the writers. Why were CNLs developed as alternatives to natural languages?
Natural language statements are often ambiguous. Consider "Sue met Tania, because she wanted to
apologize". Who wanted to apologize? Usually there is a preference to subject reference, i.e. Sue, but
there is a valid reading saying that Tania wanted to apologize. CNLs usually interdict the use of
pronouns. "Sue met Tania because Tania wanted to apologize" would be a valid CNL expression.
Natural language statements are often unspecific. Consider "The door must be opened before
entering" or "The screw driver should be placed near the screws". In both examples, the actors are not
mentioned. In technical writing, actors are essential which is why CNLs usually forbid the use of
modal verbs and of passive constructions. "Open the door before entering", or "Place the screwdriver
near the screws" are valid expressions in many CNLs.
INTERACT 611007
10
Natural language statements are often very complex. While parsing free text has made tremendous
progress, there are still constructions that cannot be parsed successfully yet. A CNL is defined in such
a way that only grammatical expressions are allowed and that these expressions can be guaranteed to
parse correctly. By the same token, translatability from one language into another one is supported by
CNLs. This is a frequent necessity by companies selling technical devices to international customers.
For computer usage, CNLs were invented since the early 1970s, with the Caterpillar Fundamental
English being a most prominent example. Boeing has a CNL to describe aircraft manuals. Another
famous CNL is AECMA Simplified English. Siemens has created a German CNL. Nothing is nown
about the detailed definitions, as these are considered proprietary knowledge by the respective
companies. An overview of the CNL topic is given by Wojcik and Hoard (1997), and with a view on
ontology editing, by Funk et al. (2007).
For INTERACT, CNLs are needed for all of the above reasons. Planners must be able to express their
intentions in a normalized, canonical fashion that can be understood easily by the workers. The
description of assembly operations is comparatively simple. While huge efforts were spent in training
writers for the Caterpillar Fundamental English, the INTERACT usage does not assume the necessity
for any language training at all. The reason for this lies in the user interface that will be adopted,
which does not allow the planner to leave the language. In composing an assembly task from parts,
the CNL description emerges. The CNL expressions on the one side and the actions, parts, tools etc.
on the other are in a bijective relationship. By using the CNL, the terminology is at the same time
unified across the company, as the CNL forces writers to use the same term for an item, and the same
phrase for an action.
With acceptable effort, these CNLs can be transposed to other natural languages, which is, however,
not part of the project INTERACT, but certainly within the space of necessities for the transnationally
active pilot case partners Daimler and Electrolux.
INTERACT 611007
11
4. BASELINE AND RESULTS ASSESSMENT
As the previous chapter gave a description on how end users work with digital human models, this
chapter has the purpose of describing the performance of the status quo digital human modelling
software EMA (editor for human work activities). EMA will be used as a baseline for the
measurement of the performance of the software prototype, which will be developed in INTERACT.
Therefore, this chapter will give a brief impression of the features and work flow of EMA. To
measure the performance of the developed prototype, parameters of performance will be defined.
These parameters will be measured for the INTERACT prototype and the EMA version of January
2014. At this point it is important to state, that the performance will be evaluated in the frame of the
defined use and pilot cases. Therewith, the measured performance only compares the performance for
these cases and their explicit features and characteristics.
4.1. EMA
4.1.1. Introduction
The EMA software is a holistic 3D planning tool for the simulation of human work activities in
industrial production. A key concept of EMA is the approach of self-initiated motion generation by
the human model, which decreases the effort for users in simulation preparation and tries to increase
the validity of simulation results in terms of realistic motion trajectories and biomechanical
correctness. This is primarily done by changing the traditional method of motion simulation using a
step-by-step animation procedure into a task- and object-oriented approach with parameterized human
movements that are calculated with generic algorithms. EMA is already capable of reproducing most
of common work-related activities. Key features of the EMA software are the implemented
assessment functions, using ‘state-of-the-art’ methods like MTM (Methods Time Measurement) for
time analysis and EAWS (Ergonomic Assessment Worksheet) for ergonomics risk evaluation
4.1.2. Creating a scene
The first step of every simulation of human work activities (hereinafter ‘simulation’) is the creation of
a virtual working environment (work place or scene). EMA uses different types of geometry data
formats to produce elements for a scene:
Geometric primitives
EMA intern objects like cubes, tables, racks, cylinders, and markers which can be varied
in their dimensions at the point of creation or adapted later in a refining process.
Collada format (.dae)
open source CAD format, which is based on xml language and was developed as
exchange format for different CAD tools.
.jt format
INTERACT 611007
12
Cad format, which is related to SIEMENS. On 2012 December, JT has been officially
published as ISO 14306:2012. EMA supports .jt-files up version 8.x.
The process of layout definition is similar to current 3D simulation standards. The organization and
definition of positions and orientations can be done by alphanumerical input or interactive 3D
manipulation via 3D window. The ‘objects” workbench also includes features like the creation and
manipulation of object-structures (hierarchy), object mass, digital human models of different
percentiles, and the (in-)visibility of objects.
Figure 2: EMA workbench
4.1.3. Task/movement definition
The movement or behavior definition in EMA follows its approach of a task-oriented self-initiative
motion generation. The description of the manual process is realized by the combination of different
“tasks”, which are compiled in the “task library” (a). The tasks are defined in accordance to the
MTM-method and represent most common tasks in work activities like ‘pick object’. ‘place object’,
‘walk’, ‘use tool’, etc.. Single tasks are put into each virtual worker’s timeline (b) by drag and drop.
For example:
Task description:
“Assemble the rear mirror at the door with two screws”
Timeline:
Figure 3: EMA task description in a timeline
INTERACT 611007
13
As shown in the figure above, in EMA the assembly task is defined by eight single tasks, which
include picking the mirror, the screws, and the screwdriver. After that mirror and screws have to be
placed and the screwdriver is subsequently used to mount the screws. Walking paths are created
automatically for each task.
Every task needs the definition of different obligatory parameters by the user. ‘Picking’ for example
needs at least the information, which object has to be picked by the DHM with which hand(s) (c).
Next to this there are optional parameters for every task. Often used optional parameters are forced
body postures, lock leg movement or processing time.
Figure 4: here EMA ‘Tasks’ workbench. a. task library, b. timeline, c. task parameter
4.1.4. Assessment tools
EMA offers different assessment tools for time related and ergonomic assessment of the simulation
process.
The ergonomic assessment workbench is based on EAWS (Ergonomic Assessment WorkSheet)
(Schaub et al. 2012). EAWS is an international established assessment tool, which focuses on the
holistic work load within manual tasks. The main parameters for the analysis are body postures,
forces, loads and the frequency respectively the duration. Additionally extra points can be added for
special difficult working conditions like impulses, hot objects, or vibrations. The result of an EAWS
analysis divides into the three categories green (no need for action), yellow (intermediate risk), and
red (immediate need for modification of the work place).
a c b
INTERACT 611007
14
The second assessment tool is the so called spaghetti diagram, which allows analysis of walking
paths. The analysis includes distances, times and the relative time at several work stations.
Figure 5: EMA spaghetti diagram with two workers
Next to these assessment tools, there is a time and added value based assessment. In these tools the
user gets basic information about time, distances and added value for every task. The results are
presented in a numerical and graphical manner.
4.2. Parameters & Methods for software evaluation
The idea behind a baseline definition is a reliable and explicit description of the status quo. To reach
that goal different aspects have to be taken in mind. First of all it is necessary to define parameters
and the corresponding methods for measuring them. The parameters are derived from the requirement
list on one hand of and on the other hand from the pilot cases, which define the frame of the aspired
performance and features of the INTERACT prototype.
Performance parameters
The first category of evaluation is the performance. Performance parameters in this case are technical
parameters in the areas of graphics and calculation time.
Quality of Results
The quality of the simulation results, as one of the main goals for development in INTERACT, is the
most difficult to determine. Since the quality of a motion in terms of style and the proximity to human
like behavior is not exclusively objective, but underlies the subjective impression of the observer.
Nonetheless it is necessary to define objective measurable parameters next to empirical user ratings,
gathered from questionnaires. These measurable parameters are defined in chapter 4.4.
INTERACT 611007
15
Usability
Usability is an important aspect of the quality of software. The three key aspects of usability are
effectiveness, efficiency and satisfaction. While effectiveness and efficiency can be measured by
certain parameters, satisfaction is difficult to determine. Beside this, one should consider, that the goal
of INTERACT is the development of a prototype and not a marked-ready software product and within
that the usability is not a key goal.
Therefore, the assessment of the usability will be limited to a comparison of the methods, movement
definition, motion refinement and the process of motion capturing. Usability related parameters like
response time are evaluated in the performance chapter and should not be evaluated twice. Especially
the motion definition and interactive changing is an important requirement by the industrial partners
and a major subject in the use cases. Within these issues it should be considered, that all potential user
groups have to be evaluated, considering their different level of expertise.
Features
The last evaluated criteria is a list of features, which is generated from the list of requirements for the
INTERACT prototype. The Baseline software EMA and the INTERACT prototype will be tested
against this list, whether they include the different features or not.
The vision of the project is to provide means to automatically generate a model for the manual
operations. Indicatively, with the software Delmia V5, the time to generate a 3D simulation model for
one station was one to two weeks for a user who has received trained in Delmia V5 but not an expert
in the Human module.
With the software EMA by imk automotive, modelling time could be reduced to 2 days with 2 expert
users from imk automotive. However, the resulting models are less realistic – especially considering
hand movements. For both Delmia V5 and EMA V5, the computer needs around 20-30 GB of RAM
to process all the data.
4.3. Success criteria
In the table below the list of criteria that will be used for accessing the success of the 3D human
simulation generation and interaction are presented. The methods to assess those criteria are also
discussed.
No Success Criterion Type Measures Target Objective
1 Time spent to develop/build a digital human simulation model for work activities (assembly tasks, materials handling) when developing a
Usability Modelling time spent per simulated time
50% - 70% reduction compared to current industrial practice (based on Delmia V5 Human Task
INTERACT 611007
16
simulation model “from scratch” (e.g. in the occasion for process planning of a new product or product variant) and get a first verified DHM based simulation of the process (1st simulation run).
Simulation) which is ~ 1 person week per simulation minute
AND
At least as good as EMA which is ~ 2-4 person days per simulation min
2 Time required for manually updating a 3D human simulation model that improves the process design following or in parallel to a prototype production phase.
Usability Modelling time spent per simulated time.
At least as good as EMA which id 15-30 minutes per simulation minute
3 Integration of Motion Captured data to 3D human simulation models during collaborative sessions with limited time constraints
Usability Time required updating a 3D human simulation with Motion Captured data.
2-3 changes in the order of the worker tasks recorded during the pilot workshops AND
4-5 changes in the process parameters recorded during the pilot workshops
4 Alternative scenarios should be planned by workshop participants. The definition of user initiated scenarios that describe alternative process plans during on-line collaborative workshops with strict time limitations
Usability Number of user initiated scenarios in on-line collaborative workshops with strict time limitations
2-3 user initiated scenario changes during process review workshops. Currently, no changes are possible in the simulation model due to the complexity to implement them.
5 Adequacy of 3D simulation for performing digital simulation analysis
Quality Objective estimates based on several criteria (see chapter below). The objective assessment will be validated based on questionnaire for assembly verification workshop participants.
Should be at least as good as a DELMIA v5 approach and considerably better than EMA
6 Calculation time required to generate a new 3D human simulation.
Performance Computation time required for the generation 1 sec of simulated time.
Maximum 5 times for calculating changes.
Table 2: Success criteria measurement
INTERACT 611007
17
4.4. Assessing the adequacy of 3D human simulation models for process
verification in the industry
Simulation models that are based on DHM are used for the verification of assembly processes in
manufacturing industry. However, currently the adequacy of such models for the needs of process
(such as assembly) verification can only be subjectively assessed by empirical user ratings, gathered
from questionnaires. The purpose of this paragraph is to propose a framework that can objectively
rate the adequacy of a DHM based simulation model for process verification purposes. This
framework can be used to objectively compare the models generated by different DHM based
simulation methods or tools or it can be used to support that the use of a simulation model is at least
as valid as the use of a physical prototype that is built for the same purposes. An important factor that
should be considered when rating the adequacy of simulation model is the role of the person that is
interested in some verification aspect of an assembly process. Typical roles are ergonomist and task
engineer (someone who assesses the performance).
The objectives is to implement those measures and their calculation method within INTERACT and
validate them by comparing subjective assessment of field experts within INTERACT industrial
partners.
No Measure Name Measure description and rationale
Calculation Method
1 Collisions (penetration greater than 5 mm)
Hypothesis: The more collisions exist during a simulation the less likely it is that this simulation is valid for validation and assessment of a process.
%of simulation time that is collision free
2 Simulation realism Hypothesis: The less realistic a simulation looks like the less useful it is for validation and assessment of a process.
Rather complex measure is
required here:
Minimum jitter or
maximum smoothness in
movement.
Use dynamics/physics to
assess a simulation
generated based on
kinematics (e.g. center of
DHM mass) (DeMagistris
et al. 2013).
Body limbs profile (e.g.
hand movements have a
bell-shaped speed profile
in straight reaching
movements) (DeMagistris
INTERACT 611007
18
et al. 2013).
Head movement (e.g.
looks at part when picking
a part)
3 Spatiotemporal constraints satisfaction
Hypothesis: If one or more process constraints (e.g. pick tool located in x,y,z within a time period) are not satisfied during the DHM based simulation the less valid this simulation is for process verification purposes.
Check list:
All parts/tools described in the work tasks are picked and placed correctly.
4 Process execution Hypothesis: If a DHM based simulation of an assembly process contradicts the process description then the simulation is unusable for process verification.
Comparison (in %) of correctly simulated process sequence to planned process sequence.
5 Precision in scene/objects representation
Hypothesis: Real world items usually vary in terms of dimensions, placing in 3d space etc. from the CAD models that were created during the design stage. A measurement of these deviations may be needed to provide an index of how well the model represents the actual scene that it simulates.
Geometric variations of each object
Placement deviation of all objects in all 6 DOFs
6 Completeness Hypothesis: If a DHM based approach for process verification cannot simulate all the process steps then this simulation is less usable for process verification.
Percentage (%) of process steps that have been simulated by a DHM approach and are clearly visible.
Table 3: Measures for adequacy of 3D human simulation models for process verification in the
industry
INTERACT 611007
19
5. REQUIREMENTS ANALYSIS
5.1. Introduction
Manual assembly simulations can significantly differ from the resulting assembly line operations they
depict. These differences, however, are not yet methodically evaluated and used in such a way that
could feed knowledge back to the simulation environments. INTERACT will utilize sensor data
coming from assembly line operations and their difference with the planned, simulated ones, in order
to achieve:
Improvement of the realism of the digital models of manual assembly operations, as well as
increased confidence in the simulation results.
Improvement of the performance of the planned assembly processes (for example in terms of
ergonomics, throughput and utilization).
Reduction of the time required to build digital models of assembly processes.
5.2. Vision and Key Features
Here we detail some important features that are required to fulfil user needs in their work as described
in the previous chapter.
5.2.1. 3D DHM level
Realistic movements
The movements should be simulated on a realistic basis, corresponding to human normal behaviour.
For instance, simulation of picking operation must be realistic time wise. Motion Captured data
taking with human subjects shall be combined with movement synthesis algorithms to model the large
range of movements to be studied during the INTERACT project.
Using collision detection, DHM shall support obstacle avoidance during complete or partial body
movement by suggesting realistic movement to avoid the obstacle(s). DHM movements must be
continuous and smooth
Motion synthesis
DHM shall support the definition of walk paths and their related postures. The motion synthesis
method should be extensible, to allow simulation of new movements. A clear methodology should be
given to update the motion synthesis capabilities with new motions (e.g. through Motion Capture.).
This is mandatory since the movements that will be studied during the project will cover only a small
part of the users' needs. Finally, multiple workers shall be possible to be simulated in sequential
operations. Parallel operations must be simulated, e. g. picking and walking, walking and turning
INTERACT 611007
20
Introduction of forces
Human motion simulation model shall be able to simulate the presence of forces. The physical
aspects of the objects shall be taken into account. For instance, forces and their impact on workers
when performing certain activities (e.g. clipsing) should be taken into account.
Task variability
Modification of anthropometrical characteristics shall have effect on postures and movements.
Definition of postures and movements shall also depend on environmental, task, spatial and temporal
constraints.
Ergonomic Assessment
The solution has to evaluate ergonomic aspects related to postures and movements.
This evaluation will be based on conventional methods like EAWS, NIOSH, reachability and
ergonomic landscapes. Ergonomic assessment should be done on a dynamic simulation and not just
static postures basis.
5.2.2. Design engineer/process planner expert level
Model generation and interaction
An initial automatically generated human simulation model from textual process description. The
user should be able to modify several aspects of the 3D DHM simulation model.
Usability in defining alternative movements
Definition of alternative task scenarios and movements shall be independent from the experience of
the operator. The user should able to define alternative task execution scenarios in very short time.
5.2.3. Ergonomics expert level
INTERACT simulation should be possible be integrated to Ergonomics methods to perform
ergonomics analysis on the simulated motion.
5.2.4. Software integration
INTERACT system should realize data retrieval from CAx systems via JT for geometric data and
structured interfaces (e.g. XML based) for alpha-numeric data.
5.3. DHM motion generation and interaction system requirements
5.3.1. Elementary motions to be simulated
In this paragraph the elementary motions that should be simulated and synthesized by the INTERACT
simulation module to be developed in WP2 “Best-fit simulation” are described. The elementary
INTERACT 611007
21
motions have been derived by analyzing the three pilot cases of the project. The elementary
movements have categorized into the following categories:
Pick (Grasp) is the motion used when the purpose is to gain control of one or more objects. In
most case a pick movement is followed by a move. It attached hand(s) to object
Release is the relinquishing of control of an object by the hand or fingers. Detach hand from
object.
Insert is the placement of a component inside another
Attach (or engage) is to connect objects.
Detach (or disengage) is the basic element used to break contact between one object and
another. Disconnect objects
Move object the predominant purpose is to transport an object to a destination.
Touch object with the end-effector.
Turn is a movement that rotates the hand, either empty or loaded. The movement rotates the
hand, wrist, and forearm about the long axis of the forearm.
Look at changes the head orientation.
Use Tool is a movement for performing operations with tools.
Body Position (Motion) are motions of the leg-foot, horizontal torso motions, and vertical torso
motions. Values include stand, sit (straight/slumped), squat, kneel (one/twoknees), lay
(prone/supine/side), crawl, bend and walk.
No. Movement
Category
Description Elementary
movement
Middle
Console
Rear-
light
Ware-
house
1 Pick
(Grasp)
Attach hand(s) to object Pick-up using right/left
hand or both
× × ×
2 Pick
(Grasp)
Attach hand(s) to object Transfer (control
transfer grasp from
one hand to other)
×
3 Pick
(Grasp)
Attach hand(s) to object. Regrasp (Change
grasp without
relinquishing control)
×
4 Pick
(Grasp)
Attach hand(s) to object Contact Sliding, or
Hook Grasp
5 Release Detach hand from object Release right/left hand
or both
× × ×
6 Insert Place component inside
another
One hand moving
towards the other, both
holding components
× ×
7 Attach Connect objects Attach/Connect/Engag
e objects
× ×
8 Detach Disconnect objects Disengage ×
9 Move In Move object, the
predominant purpose is
Push/pull object × ×
INTERACT 611007
22
Table 4: List of motions to be simulated in INTERACT
5.3.2. Derivation of a sequence of elementary motions from planning texts
The definition of a sequence of motion elements that execute the task will be achieved through CNL
that will map the description of tasks to motion elements. Assembly descriptions based on a
Controlled Natural Language (CNL) have a significant advantage over those formulated without
restrictions – they can easily and unambiguously be parsed, processed and generated. The CNL
system enforces the use of controlled language and therefore makes it possible to automatically
process and analyze assembly descriptions.
object to transport an object to
a destination.
(extend / retract arms)
10 Move
object
In Move object, the
predominant purpose is
to transport an object to
a destination.
Carry (move body to
change position while
holding object)
× ×
11 Move
object
In Move object, the
predominant purpose is
to transport an object to
a destination.
Press/Lift (move
object down/up)
× ×
12 Touch Touch object with the
end-effector
Apply pressure with
finger
13 Turn Turn is a movement that
rotates the hand, either
empty or loaded.
The movement rotates
the hand, wrist, and
forearm about the long
axis of the forearm
(left/right).
× ×
14 Look at Change head orientation Turn head to look
around
× ×
15 Use Tool Use battery tool Loosen/Tighten nut × ×
16 Use Tool Use cordless
screwdriver/tool
Bolt-
Connect/Disconnect
×
17 Use Tool Rivet gun Hammer rivets ×
18 Touch Use a machine’s controls Press button
(left/right) hand
×
19 Body
Position
Vertical Motion Kneel (one leg or
both)
20 Body
Position
Vertical Motion Bent forward
21 Body
Position
Horizontal Motion Side step ×
22 Body
Position
Horizontal Motion Walk unobstructed and
without carrying loads
× × ×
INTERACT 611007
23
The CNL descriptions of assembly activities come at different grain sizes. In any case, an assembly
shall be represented by a sequence of CNL expressions in their temporal order. Each CNL expression
is analyzed (see below). The analysis consists of linguistic analysis on the one hand and enrichment
with parameter values on the other.1 For ease of reference let us call the formal language, in which
analyses are represented, SD (for Semantic Descriptions).
Each SD expression can be mapped onto a sequence of motion elements that are used to visualize the
intended meaning. More formally, a homomorphism from SD expressions onto sequences of motion
elements shall exist.
Figure 6: Homomorphic Mapping (grey) of SD Expressions (green) Onto (Sequences of) Motion
Elements (orange)
Figure 6 depicts a situation in which for each of the SD expressions a sequence of motion elements
has been found. The borders between the adjacent sequences need to be merged, unless mergers from
the past can be reused. The merging results depend on the supplied parameter settings; they will be
stored for future reuse.
The merging is context-dependent. Therefore the interface from SD to the selection of motion
elements consists of a sequence of two SD expressions, namely the current one to be mapped, and the
subsequent one, which the current one must merge with. This can be seen like a window shifted over
the sequence of events, when the material “left behind” is already visualized, and the material “ahead”
consists of CNL statements.
Requirement Description
Automatic derivation of a sequence of elementary
motions from planning text.
The INTERACT system should be able to derive
the sequence of elementary motions (e.g. picking,
walking, turning etc.) from initial textual
description plus dynamic and static role filler
properties as constraints over motion sequences.
Table 5: Requirements for derivation of a sequence of motions
1 To be specified. For instance: 3D location, weight, grasp points.
INTERACT 611007
24
5.3.3. Motion constraints generation
The INTERACT system should be able to define the motion constraints (e.g. what part to pick, where
to place it, using left or right hand etc.) by utilizing data coming from CAD, CAPP system regarding
the process that should be simulated.
Some CNL expressions are underspecified with regard to the properties of the participating elements.
Each element (e.g. “cordless screw driver”, or “middle console”) has parameters such as weight, grasp
points etc. The values of these parameters must be available, with a unique index pointing from the
ontology to the data.
Similarly, some CNL expressions are underspecified with regard to details of the activities. For
instance, the activity of picking up screws can be carried out by picking one screw at a time, or all
screws at once. Screws can be carried in one hand, in both hands, or in the trouser pocket (other
manners possible). The CNL must explicitly mention any non-retrievable details of movements that
pertain to the selection of motion elements.
Motion synthesis should take into account the environment in the sense that collisions with object in
the environment and the animations of the avatar are avoided. In general, collision avoidance would
need motion planning with a path planning approach that computes collision free movements for the
avatar taking into account all individual joints the avatar might have. However, for an avatar with a
high number of joints such an approach is computational expensive. Depending on what kinds of
movements are needed more efficient solutions are possible. If for example the avatar needs to walk
from one position to another in an environment which is not crowded with obstacles, it is possible to
compute a navigation mash which consists of trajectories which allows the avatar to move without
worrying about collisions if its centre of mass is following the trajectory accurately enough. To
achieve this the avatar is usually put into a bounding box or into a cylinder and computes paths which
allows this bounding box or cylinder to move without collision from one position to another one.
While moving the avatar is then allowed to act freely in this bounding box or cylinder. The situation
gets more complex in case the environment contains moving objects itself. However, if the
movements of these objects are not too complex they can be taken into account.
Apart from the constraints that are inferred by analysing the environment the user should also be able
to modify or explicitly define constraints (e.g. define a specific point in a walk path).
Requirement Description
Automatic inferring of motion constraints from The INTERACT system should be able to
INTERACT 611007
25
product, process and resource data and a-priori
knowledge
automatically define the motion constraints by
utilizing data coming from CAD, CAPP, work
task information including descriptions, relevant
parts, worker assignment, worker position.
Collision free DHM walks The motion planning subsystem should generate
movements of the DHM that are collision free.
Explicit definition of a constraint in the 3D space The user should be able to explicitly define new
constraints in a motion or to explicitly change an
inferred constraint.
Table 6: Motion constraints generation requirements.
5.3.4. Motion synthesis
The INTERACT human motion simulation system should be able to smoothly combine primitive
human motion in order to generate new motions. This will allow the concatenation of primitive
movements to form a rich repertoire of human activities. Moreover, the motion synthesis method
should be extensible, to allow simulation of new movements. A clear methodology should be given to
update the motion synthesis capabilities with new motions (e.g. through Motion Capture.). This is
mandatory since the movements that will be studied during the project will cover only a small part of
the users' needs. Motion synthesis capability will provide the backbone to enable the user to
accurately and interactively control a DHM by simply issuing high-level control commands such as
“walking to reach a point and “picking up the object at”.
Moreover, it should be able to generate and simulate parallel operations, e. g. picking and walking,
walking and turning.
The requirements for motion synthesis are listed in the following table:
Requirement Description
Combine human motions in order to generate
new motions
The motion generation system should be possible
to combine several motions and generate a big
repertoire of motions.
Motion adaptation to new constraints The INTERACT motion generation sub-system
should be able to adapt a motion to new motion
constraints.
Change the DHM
Change the environment
Change both DHM and the environment
Motion generation capability should be
extensible to a numerous types of motions
The motion generation system should be possible
to be expanded and include other types of
INTERACT 611007
26
motions that may not be included in the initial list
defined by the project case studies.
Generation and simulation of parallel operations
of one worker
It should be possible to generate motions that
execute parallel operations e. g. picking and
walking, walking and turning.
Table 7: Motion synthesis requirements
5.3.5. Model interaction
5.3.5.1. CNL based interaction
The system described in (Manns and Arteaga, 2013) enables the planner to express assembly activities
in CNL. Based on the activity, the system presents to the planner a template with placeholders for its
possible roles. Some roles are optional, others are obligatory. For each role, the system also presents
a set of reasonable filler parts. These filler parts are extracted from various external data sources. The
planner selects suitable fillers, and the CNL text is generated, as planning proceeds. By virtue of the
CNL, the planner cannot accidentally generate uncontrolled text. Both templates and filler parts are
available in several languages, which allows for the automatic translation of the assembly activity
descriptions into any other supported language. The current working hypothesis is that a similar
interface is suitable for INTERACT as well. By and large this depends on the complexity of the CNL
required.
5.3.5.2. Sensor based interaction
The INTERACT simulation model should be editable though the sensor system. A user should be
able to execute a task or a portion of a task (i.e. some steps) within a motion capture area and the
captured motion should be used to update/edit the DHM based motion simulation. For example an
engineer may want to assess a ‘what-if’ scenario in which a different tool type (than the planned one)
is used and the tool is located in a different location than the one planned. If the new tool type is
available in the motion capture area the user may execute and capture the portion of the whole task
that is related to the use of the new tool type (and new tool location). Then the INTERACT system
should be able to update the originally planned, simulated motion with key-aspects (e.g. new tool
location, new tool type, probably new working posture using the new tool) of the captured motion.
The update should be possible through utilizing the motion synthesis and generation functionality.
Further, details on how the INTERACT sensor system can be used to provide input to motion
synthesis is given in INTERACT D1.2.1 chapter 5.1.1 and 5.1.2.
INTERACT 611007
27
5.3.5.3. Change order of work tasks
The INTERACT simulation model should be editable by modifying (aka add, remove or change the
order) of work task. A user should be able to change the execution order of work tasks and the
INTERACT simulation system should be able to simulate the new work task execution order or to
report exceptions and inconsistencies.
Requirement Description
CNL based interaction The user may modify terms in CNL descriptions.
Sensor based interaction The user may initiate some Motion Capture session (using sensor
technology to be defined in INTERACT WP4). The new captured
motion, which corresponds to some specific task, can be used to
update/edit the DHM based motion simulation.
Change order of work tasks The user should be able to change the order of work tasks and this
should be depicted in the simulated motion.
Table 8: Requirements for interacting with the DHM based process model.
5.3.6. Motion simulation / animation
The INTERACT system apart from motion synthesis and generation should be able to
simulate/animate a generated motion. The INTERACT simulation subsystem should implement the
following requirements:
Requirement Description
DHM motion simulation/animation The INTERACT simulation subsystem should be able to
simulate a DHM motion
Scene objects (not only DHM) should
able to change during simulation
Objects in the scene (e.g. parts, tools) should move during
simulation.
Objects moving along with DHM (e.g. worker
carrying a part)
Objects moving on the occurrence of some event
(e.g. worker presses a button and a part rotates)
Object moving when the object that it is attached to
is moving (e.g. part on table moves when the table is
moving)
Work step recognition during 3D
simulation
During simulation it should be possible to recognize
(through annotation) the work steps executed by the DHM.
Table 9: Motion simulation/animation requirements
INTERACT 611007
28
5.3.7. Ergonomics assessment
The INTERACT system should be able to provide ergonomic assessment of existing DHM
simulations. It should provide ergonomic assessment functionality for the EAWS, NIOSH and OCRA
standards. More details on the ergonomics assessment requirement are provided in the Ergonomic
App in INTERACT deliverable D1.3.1.
5.3.8. Non-functional requirements
In terms of system performance and quality, several important factors are relevant for the motion
generation and subsystems of the INTERACT system. See the following table for a list of non-
functional requirements the system has to implement.
Requirement Description
Motion must be smooth The generated motion should be smooth without
jerkiness or sudden locomotion or body limbs.
Realistic human motion simulation Human motion must be realistic incl. acceleration
and deceleration behavior
DHM walk paths must be continuous and smooth There should be no human body sudden
locomotion. The walking pattern (e.g. make a
side step after having walked forward) should
change smoothly.
DHM simulation must be changeable on-the-fly The simulation model should be editable and re-
generated on the fly so as it may support on-line
collaborative sessions.
Table 10: Non-functional requirements
INTERACT 611007
29
6. CONCLUSIONS
The primary conclusions/results include the following:
The main result of this deliverable is the list of requirements and key features for the INTERACT
motion simulation developments to take place in WP2 “Best-fit simulation of manual assembly
using sensor data”.
The analysis of the state of the art has indicated that there is a high potential in using the
principles of the morphable graphs approach for a baseline in DHM simulation.
The list of success criteria will be used in WP 6 “Industrial Pilot Cases, Demonstration and
System Validation” for assessing the performance of INTERACT motion generation and
synthesis approach.
The requirements defined in this document will serve as an input to the RTD tasks of WP2 “Best-fit
simulation of manual assembly using sensor data”.
INTERACT 611007
30
7. GLOSSARY
DHM Digital Human Modelling
PLM Product Lifecycle Management
MMT Material Management Team
VMS Virtual Manufacturing Software
RULA Rapid Upper Limb Assessment
MOST Maynard Operation Sequence Technique
AAWS Automotive Assembly Worksheet
CAPP Computer Aided Process Planning
CAD Computer Aided Design
C-values C-Values are a Daimler in-house system that
has been developed from MTM. Objectives are
reduction of analysis time and application
throughout the planning phase.
eHPV Engineered Hours Per Vehicle
MV Manufacturing Variable
INTERACT 611007
31
8. REFERENCES
1. Michalos, G., Makris, S., Papakostas, N., Mourtzis, D., Chryssolouris, G.: Automotive
assembly technologies review: challenges and outlook for a flexible and adaptive approach. CIRP
Journal of Manufacturing Science and Technology 2(2), 81–91 (2010)
2. Chryssolouris, G., Manufacturing Systems: Theory and Practice, 2nd Edition, Springer-
Verlag, New York, New York, (2006)
3. Manns and Arteaga 2013, Automated DHM Modeling for Integrated Alpha-Numeric and
Geometric Assembly Planning, M. Abramovici and R. Stark (Eds.): Smart Product Engineering,
LNPE, DOI: 10.1007/978-3-642-30817-8_32, pp. 325–334
4. Siemens 2014,
http://www.plm.automation.siemens.com/en_us/products/tecnomatix/assembly_planning/jack/
5. Dassault Systemes 2014, http://www.3ds.com/products-services/delmia/
6. K. Alexopoulos, D. Mavrikios, G. Chryssolouris, "ErgoToolkit: an ergonomic analysis tool in
a virtual manufacturing environment", International Journal of Computer Integrated Manufacturing
Vol.26, No.5, pp.440-452 (2013)
7. Härtel, T. ; Keil, A.; Hoffmeyer, A.; Toledo Munoz, B.: Capturing and Assessment of Human
Motion during Manual Assembly Operation, First International Symposium on Digital Human
Modeling, Conference Proceedings, June 14-16 2011, Lyon, France.
8. Fritzsche, L., Jendrusch, R., Leidholdt, W., Bauer, S., Jäckel, T., Pirger, A.: Introducing EMA
(Editor for Manual Work Activities) – A New Tool for Enhancing Accuracy and Efficiency of Human
Simulations in Digital Production Planning. In: Duffy, V.G. (ed.) ICDHM 2011. LNCS, vol. 6777, pp.
272–281. Springer, Heidelberg (2011)
9. Gläser, D.; Fritzsche, L.; Bauer, S.; Leidholdt, W., The quest to validated human motion for
digital ergonomic assessment, AHFE 2014 – conference Proceedings, Krakow (2014)
10. Schaub, K., Caragnano, G., Britzke, B., Bruder, R. (2012): The European Assembly
Worksheet. Theoretical Issues in Ergonomics Science, 27 April, No. 1.
11. Giovanni De Magistris„, Alain Micaelli„, Jonathan Savin…, Clarisse Gaudez…, Jacques
Marsot, Dynamic Digital Human Model for ergonomic assessment based on human-like behaviour
and requiring a reduced set of data for a simulation, 2013 DIGITAL HUMAN MODELING
SYMPOSIUM.
12. Mavrikios, D., Pappas, M., Kotsonis, M., Karabatsou, V., Chryssolouris, G., "Digital Humans
for Virtual Assembly Evaluation", Digital Human Modeling, Duffy V.D. (ed), Volume 4561, pp. 939-
948, Springer-Verlag (2007)
INTERACT 611007
32
13. Pappas, M., Karabatsou, V., Mavrikios, D., Chryssolouris, G., "Ergonomic evaluation of
virtual assembly tasks", Digital Enterprise Technology: Perspectives and Future Challenges, P.
Cunha, P. Maropoulos (eds), Session 5, pp. 511-518, Springer-Verlag (2007)
14. Chryssolouris, G., V. Karabatsou and G. Kapetanaki, "Virtual Reality and human simulation
for manufacturing", Proceedings of the 34th International CIRP Seminar on Manufacturing Systems,
Athens, Greece, (May 2001), pp. 393-398.
15. Park, W., Chaffin, D.B. and Martin, B.J., Toward memory-based human motion simulation:
development and validation of a motion modification algorithm. IEEE Trans. Systems, Man,
Cybernetics, Part A: Systems and Humans, 2004, 34, 376–386.
16. Bruderlin, A. and Williams, L., Motion signal processing. Proceedings of SIGGRAPH, Los
Angeles, California, 1995, pp. 97–104.
17. Witkin, A. and Popovic, Z., Motion warping. Proceedings of SIGGRAPH, Los Angeles,
California, 1995, pp. 105–108.
18. Gleicher, M., Retargeting motion to new characters. Proceeding of ACM SIGGRAPH
Conference, 1998, pp. 33–42.
19. Yang, J., R. T. Marler, H. Kim, J. Arora, and K. Abdel-Malek, “Multi-objective optimization
for upper body posture prediction”, Proceedings of the 10th Multidisciplinary Analysis and
Optimization Conference (AIAA/ISSMO), Albany, N.Y., American Institute of Aeronautics and
Astronautics (AIAA), 2004.
20. Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion Graphs. In ACM Transactions on
Graphics. 21(3):473–482.
21. Safonova, A., and Hodgins, J. K. 2007. Construction and optimal search of interpolated
motion graphs. In ACM Transactions on Graphics. 26(3).
22. Heck, R. and Gleicher, M. 2007. Parametric motion graphs. In 2007 ACM Symposium on
Interactive 3D Graphics.
23. Min,J., Chai, J. Motion graphs++: a compact generative model for semantic motion analysis
and synthesis. ACM Trans. Graph.31(6) 2012.
24. Demirel O. and Duffy V., 2007. Applications of Digital Human Modeling in Industry. In:
V.G. Duffy (Ed.), Digital Human Modeling, HCII 2007, LNCS 4561, 824–832.
25. Rhen, I.M., Gyllensvärd, D., Hanson, L. and Högberg, D., 2011a. Time dependent exposure
analysis and risk assessment of a manikin’s wrist movements. Proceedings of DHM 2011, First
International Symposium on Digital Human Modeling, France, ISBN 978-2-9539515-0-9.
INTERACT 611007
33
26. Rhen, I.M., Hanson, L. and Högberg, D., 2011b. Risk exposure assessment of dynamic wrist
motions of a digital human model. Proceedings of the 43rd annual Nordic Ergonomics Society
Conference, Oulu, Finland, September 2011, ISBN 978-951-42-9541-6.
27. Ma, L., Zhang, W., Fu, H., Guo, Y., Chablat, D., Bennis, F., Sawanoi, A., and Fugiwara, N.,
2010. A Framework for Interactive Work Design Based on Motion Tracking, Simulation, and
Analysis. Human Factors and Ergonomics in Manufacturing & Service Industries, 20(4) 339–352.
28. Winter, G., Schaub, K., and Landau, K., 2006. Stress screening procedure for the automotive
industry: development and application of screening procedures in assembly and quality control.
Occupational Ergonomics, 6, 107–120.
29. Wojcik, R., & Hoard, J. (1997). Controlled Languages in Industry. In R. Cole, J. Mariani, H.
Uszkoreit, G. Varile, A. Zaenen, A. Zampolli et al. (eds.), Survey of the State of the Art in Human
Language Technology, (pp. 238-239). Cambridge: Cambridge University Press. Online available at
http://www.lt-world.org/hlt-survey/master.pdf
30. Funk, A., Tablan, V., Nontcheva, K., Cunningham, H., Davis, B. & Handschuh, S. (2007).
CLOnE: Controlled Language for Ontology Editing. In K. Aberer et al., (eds.), The Semantic Web.
6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, ISWC 2007 +
ASWC 2007, Busan, Korea, November 11-15, 2007. Proceedings, Springer: Berlin, Heidelberg, pp.
142-155.
INTERACT 611007
34
9. APPENDIX I: USER REQUIREMENTS CATALOGUE FOR DHM
BASED MODEL SIMULATION
Need Category
1 An ergonomy landscape has to be derivable from a 3 D simulation scenarios
ergonomic assessment
2 Assembly operations must be able to blend into each other (no return to base pose)
quality of results
3 Assembly operations must be smooth quality of results
4 Assembly sequence must not contradict itself quality of results
5 Assessment functionality for space requirements for station as in 3D scene
assessment general
6 Collision avoidance accuracy < 5 mm features
7 Comfortable work area for a worker must be visualizable ergonomic assessment
8 Containers must be able to move with the car (be positionable in moving coordinate system)
features
10 DHM MV processes (incl. walking) must be realistic incl. acceleration and deceleration behavior
quality of results
11 DHM MV processes (incl. walking) must be realistically timed incl. acceleration and deceleration behavior
quality of results
12 DHM MV processes (incl. walking) must look realistic quality of results
13 DHM simulation must be changeable within the workshop by each participant
usability
14 DHM walk paths must be collision free quality of results
15 DHM walk paths must be continuous and smooth quality of results
16 Each work step must be recognizable in the 3D simulation quality of results
17 Ergonomic evaluation of pick operations from part carriers with different filling percentage
ergonomic assessment
18 Ergonomic evaluation: EAWS criteria ergonomic assessment
19 Ergonomy landscapes have to be comparable for multiple scenarios ergonomic assessment
20 Error handling if a planning text does not match the controlled natural language
usability
21 Interactive comparison of planned MTM-1 bundles to the ones derived from the simulation (maybe with an automated comparison tool that displays deviations)
assessment general
22 Interactive reachability analysis ergonomic assessment
INTERACT 611007
35
23 Interactive user selection of screw or clip usability
24 Movement generation of a DHM so that the hand or tool reaches a selected tool or clip
quality of results
25 MTM 1 bundle scenarios must be comparable assessment general
26 MTM 1 bundles (C-Values) have to be derivable from the simulation model
assessment general
27 Multiple workers must be simulated in sequential operations features
28 Parallel operations must be simulated, e. g. picking and walking, walking and turning
quality of results
29 Part carriers have to be movable in the 3D scene in the workshop features
30 Part orientation operations must look realistic quality of results
31 Picking operations must look realistic quality of results
32 Planned MTM 1 bundles (C-Values) must be interactively changeable in the work shop
assessment general
33 Realistic simulation of one point picking (for larger parts) for assessment of feasibility
quality of results
34 DHM simulation scenarios must be comparable within the workshop features
35 Simulation scenarios have to be associated with MTM 1 bundle scenarios and compared
assessment general
36 Simulation of part orientation must be realistic time wise quality of results
37 Simulation of picking operation must be realistic time wise quality of results
38 The visualisation must give feedback if there are walk way tasks that match the simulated walks
assessment general
39 The visualisation should propose addition, adaptation or elimination of inadequately planned walk paths
assessment general
40 Unrealistic worker poses must be recognizable or avoided (physical force)
quality of results
41 Visualization of view area of the DHM (part visibility) and automatic recognition of blind assembly situations
ergonomic assessment
42 Work process (eHPV and MV) times have to be realistic quality of results
43 Work task sequence must be interactively changeable by workshop participants
usability
44 Works steps in the 3D simulation must correspond to work task descriptions
quality of results
45 (New) Item handling process definition features
46 Clear visualization of hands representation and movements in item handling
quality of results
47 Enable manual modification of the proposed outcome of the item handling process definition
features
INTERACT 611007
36
48 Trolley uploading process visualization quality of results
49 Trolley design suggestion/modification features
50 Warehouse container design features
51 Warehouse shelves design features
52 Picking process simulation (one item) features
53 Picking process simulation for a complete sequence (all components per one trolley)
features
54 It should be possible to manual modify the outcome of the picking process sequence
features
55 Ergonomic assessment (NIOSH standard) ergonomic assessment
56 During ergonomics assessment (NIOSH) there should be dynamic indications with red/green lights of the correctness of the operations
ergonomic assessment