quality assesment in bone scaffolds through internet based
TRANSCRIPT
University of Texas at El PasoDigitalCommons@UTEP
Open Access Theses & Dissertations
2010-01-01
Quality Assesment In Bone Scaffolds ThroughInternet Based Robotic Using Intelligent DataMiningAditya ChilukuriUniversity of Texas at El Paso, [email protected]
Follow this and additional works at: https://digitalcommons.utep.edu/open_etdPart of the Engineering Commons
This is brought to you for free and open access by DigitalCommons@UTEP. It has been accepted for inclusion in Open Access Theses & Dissertationsby an authorized administrator of DigitalCommons@UTEP. For more information, please contact [email protected].
Recommended CitationChilukuri, Aditya, "Quality Assesment In Bone Scaffolds Through Internet Based Robotic Using Intelligent Data Mining" (2010).Open Access Theses & Dissertations. 2457.https://digitalcommons.utep.edu/open_etd/2457
QUALITY ASSESMENT IN BONE SCAFFOLDS THROUGH INTERNET BASED
ROBOTIC USING INTELLIGENTDATA MINING
ADITYA CHILUKURI
Department of Industrial Engineering
APPROVED:
Tzu-Liang(Bill) Tseng, Ph.D., Chair
Tao Xu, Ph.D.
Eric D.Smith, Ph.D.
Patricia D. Witherspoon Ph.D. Dean of the Graduate School
Copyright ©
by
Aditya Chilukuri
2010
This thesis is dedicated to my wonderful mother Smt. LalithaChilukuri
… With respect and love
QUALITY ASSESMENT IN BONE SCAFFOLDS THROUGH INTERNET BASED
ROBOTIC USING INTELLIGENT DATA MINING
by
ADITYA CHILUKURI, B.TECH (ME)
THESIS
Presented to the Faculty of the Graduate School of
The University of Texas at El Paso
in Partial Fulfillment
of the Requirements
for the Degree of
MASTER OF SCIENCE
Department of Industrial Engineering
THE UNIVERSITY OF TEXAS AT EL PASO
December 2010
v
Acknowledgements
It is a pleasure to thank those who made my thesis possible, I would like to show my gratitude to
my supervisor Dr. Bill Tseng whose encouragement, guidance and support from the initial to the final
level enabled me to develop an understanding of the research topic. He was the one who stimulated my
research interest in machine vision systems and artificial intelligence. He has spent numerous hours
with me discussing many ideas and technical aspects which eventually led to this thesis. Finally, I would
like to sincerely thank him for the financial support that he has provided me because of which I could
continuously work without any difficulties. I am grateful to Dr. Tao Xu, who helped me in choosing the
correct case study for my research and his contributions for my work. I also thank other members of the
thesis committee for their time and participation in the thesis.
I would like to thank Jorge, student of Dr.Xuwho helped me a lot in designing the case study. I
would like tomake special acknowledgements to present and past members of Intelligent Systems
Engineering Laboratory especially Prashanth, Zhonghua, Ugandhar, Kalyan and others, who have
helped me in one way or other and made my graduate study at University of Texas at El Paso a pleasant
experience. I want to thank the faculty of Industrial Engineering Department for all the guidance and
giving me the right knowledge and experience to excel in my areas of interest.
I would like to thank Almighty for all the blessings in successful completion of my thesis work.
Last but not the least, I would like to express my deepest gratitude towards my parents who inculcated
the art of learning and for their love and support throughout, my brother Anil, for his encouragement and
support and Amma who has been always with me in any kind of hardships.
vi
Abstract
Optimization of design in fabricating scaffolds in an important step in obtaining tissue engineering
scaffolds with appropriate shape and inner microstructure. Different shapes & sizes of scaffolds are
modeled using UGS NX 6.0 modeling software with variable pore sizes. The quality issue we are
concerned is about scaffold porosity, because of fabrication errors the porosity is not be very high for
few models. There is lot of research ongoing for characterization using scanning electron microscope,
but this study will introduce a new technique to characterize the scaffolds using network based
robots, machine vision system and conveyor facility using which the process can be automated.
The insight software for the Cognex camera is set such that it can determine whether the specimen can
be useful or not instantly. The purpose of this research is wholly to improve the quality at earlier stages
of manufacturing by which overall cost can be reduced and further preventing processing of defective
during manufacturing. In this paper we will analyze the data collected from fabricated scaffolds
using neural networks for classification and regression analysis and then design of experiments for
total automated diagnosis of the part mainly considering the surface features for analysis.
vii
Table of Contents
Acknowledgements .................................................................................................. v
Abstract ................................................................................................................... vi
Table of Contents .................................................................................................. vii
List of Tables .......................................................................................................... ix
List of Figures .......................................................................................................... x
Chapter 1: Introduction ............................................................................................ 1 1.1 Introduction ............................................................................................ 1 1.2 Network based quality assessment setup ............................................... 2 1.3 Research Overview ................................................................................ 5 1.4 Thesis Organization ............................................................................... 5
Chapter 2: Literature Review ................................................................................... 6 2.1 Quality assessment ................................................................................. 6 2.2 Tissue Engineering ................................................................................ 7 2.2.1 Introduction ............................................................................................ 7 2.2.2 Design requirements: ............................................................................. 7 2.2.3 Design Optimization and fabrication: .................................................... 9 2.2.4 Literature Survey for 3D Printing: ....................................................... 11 2.3Internet based robotics ............................................................................. 14 2.4 Data Mining - Neural Networks ....................................................... 14 2.4.1 Background .......................................................................................... 20 2.4.2 Network Architectures ........................................................................ 20 2.4.2.1 Single-Layer Feed forward Networks .......................................... 20 2.4.2.2 Non linearfeed forward Networks ................................................. 21 2.4.3 Applications ......................................................................................... 27 2.5 design of experiments ....................................................................... 28
Chapter 3: Methodology ........................................................................................ 29 3.1 Design Selection .................................................................................. 31 3.2 Neural network model ......................................................................... 32 3.2.1 MLP model .......................................................................................... 32
viii
3.2.2 RBF model ........................................................................................... 39 3.3 Design of experiments ......................................................................... 44
Chapter 4: Case study ............................................................................................ 46
Chapter 5: Analysis ................................................................................................ 52 5.1 Classification analysis ......................................................................... 52 5.1.1 Multi layer perceptrons ........................................................................ 52 5.1.2 Radial basis function ............................................................................ 57 5.1.3 Analysis of results ................................................................................ 62 5.2 Regression analysis .............................................................................. 63 5.2.1 Multilayer perceptron neural network ................................................. 63 5.2.2 Radial basis function neural network .................................................. 68 5.2.3 Design of experiments ......................................................................... 72 5.2.3.1 Factorial fit ..................................................................................... 73
5.2.3.2 Response surface regression analysis ............................................ 76 5.2.3.3 Response optimizer ........................................................................ 77 5.2.4 Analysis of results ................................................................................ 78
Chapter 6: Conclusions .......................................................................................... 80 6.1 Future work .......................................................................................... 82
Bibliography .......................................................................................................... 83
Appendix ................................................................................................................ 88 Appendix 1: Experimental Data ................................................................... 88 Appendix 2: Robot Program ......................................................................... 94
Vita ……. .............................................................................................................. 96
ix
List of Tables
Table 2.1: Biometric design considerations and possible design solution. ..................................... 8 Table 2.2: Currently applied 3D fabrication technologies. ............................................................ 10 Table 2.3: Review of Neural Networks related research. .............................................................. 16 Table 2.4: Comparison between Multi Layer Perceptrons and Radial Basis Function networks . 26 Table 3.1: Factors and levels of interest for Design of Experiments. ............................................ 44 Table 3.2: Design of the model with data for Design of Experiments .......................................... 45 Table 5.1: Multi layer perceptron network analysis. ..................................................................... 54 Table 5.2: Target and output decisions for MLP based on ensemle predictions. .......................... 56 Table 5.3: Radial basis function network analysis. ....................................................................... 58 Table 5.4: Target and output decisions for MLP based on ensemle predictions. .......................... 60 Table 5.5: Multi layer perceptron regression analysis predictions. ............................................... 65 Table 5.6: Radial basis function regression analysis predictions. ................................................. 69 Table 5.7: Computational results for regression analysis. ............................................................. 79
x
List of Figures
Figure 1.1: Work Area set up at ISEL, UTEP. ................................................................................ 2 Figure 1.2: Overall setting of system at ISEL,UTEP. ..................................................................... 3 Figure 1.3: Application Programming Interface. ............................................................................. 4 Figure 2.1: 3D printing step by step process. ................................................................................ 12 Figure 2.2: Model of a neuron. ...................................................................................................... 15 Figure 2.3: Diagram of a single-layer feed forward artificial neural network.. ............................. 21 Figure 2.4: Diagram of a multi-layer feed forward artificial neural network. ............................... 22 Figure 2.5: Figure showing RBF activation function and effect of distance onactivation. ........... 23 Figure 2.6: Figure showing the effect of spread on the neuron. .................................................... 24 Figure 2.7: Radial Basis Function network model. ....................................................................... 24 Figure 3.1: Conceptual framework for methodology development ............................................... 30 Figure 4.1: Screen shot of scaffold model in UGS NX 6.0. .......................................................... 46 Figure 4.2: Step by step process for complete fabrication, manufacture and inspection. ............. 47 Figure 4.3: (a) Working with Z corporation 3D printer Z450 (b) Using Z Print for setting up models for printer. (c) 3D printer making prints of scaffold models.. ........................................... 48 Figure 4.4: Screen shot of hexagonal and circular scaffolds ………………………................... 49 Figure 4.5: Screen shot of Cognex insight explorer with scaffold being investigated................. 50 Figure 4.6: Hexagonal scaffolds with varying pore size from 0.5 -1mm. ..................................... 51 Figure 4.7: Circular scaffolds with varying pore size from 0.5 -1mm. ......................................... 51 Figure 5.1: Screen shot of multi layer perceptron working with Statistica 9.1 ............................. 53 Figure 5.2: Result from multi layer perceptron neural network. ................................................... 53 Figure 5.3: Graph showing the MLP network error in each phase of analysis. ............................ 55
xi
Figure 5.4: Histogram of network type versus accuracy for MLP network. ................................. 55 Figure 5.5: Result from multi layer perceptron neural network for networks 10, 13 & 14. .......... 56 Figure 5.6: Result from radial basis function neural network. ...................................................... 58 Figure 5.7: Graph showing the MLP network error in each phase of analysis. ............................ 59 Figure 5.8: Histogram of network type versus accuracy for MLP network. ................................. 59 Figure 5.9: Screenshot showing target and output for decision with ensemble predictions. ......... 60 Figure 5.10: Porosity distribution for given data . ......................................................................... 63 Figure 5.11: Screenshot showing result window for MLP network. ............................................. 64 Figure 5.12: Graph of samples versus porosity performing regression analys for MLP network. 64 Figure 5.13: Screenshot showing result window for RBF network. .............................................. 68 Figure 5.14: Graph of samples versus porosity performing regression analys for RBF network. 68 Figure 5.15: Screen shot of Minitab 15 working on Design of Experiments. ............................... 72 Figure 5.16: ANOVA result from Minitab. ................................................................................... 73 Figure 5.17: Normal probability plot. ............................................................................................ 74 Figure 5.18: Test for equal variances. ........................................................................................... 74 Figure 5.19: Histogram of residuals versus frequency. ................................................................. 75 Figure 5.20: Residuals versus fits. ................................................................................................. 75 Figure 5.21: Response surface regression analysis. ....................................................................... 76 Figure 5.22: Response optimizer for regression analysis. ............................................................. 77 Figure 5.23: Graph of samples versus porosity performing regression analys for DOE. .............. 78
1
Chapter 1: Introduction
1.1 INTRODUCTION
In future design, manufacturing, quality needs to be merged with the information management networks.
Current trend in manufacturing is remote monitoring and control of the production systems, since
analysis is performed online, the product life cycle as a whole increases along with it the products will
have high precision. Generally robots are manufactured for high tolerance standards, and the control and
feedback devices are technically advanced so that we get motion and path controls, in spite of having
many advantages they suffer from errors like geometric errors, joint alignment, dynamic errors and
thermal errors. Garvin [1] defines five different quality perspectives, among which two have been
widely adopted when defining quality models. The first one is ‘quality as the degree of compliance with
respect to certain specifications’ and second being Quality as ‘meeting customer needs’. Even if much
more complex to evaluate, it is this second perspective the one that, standards, should make up the
overall objective of any quality evaluation process. Starting to assess quality at such a late stage of
development is avowed to have a negative impact on the final product cost and quality. In fact,
according to Moody the cost associated with removing a defect during design is on average 3.5 times
greater than during requirements. Other empirical studies [1] have shown that moving quality evaluation
effort up to the early phases of development can be 33 times more cost effective than testing done at the
end of the development. This assumption means that is it possible to improve the web quality in use by
working on the quality of each outgoing artifact and this all discusses about the quality control use at the
design level.The recent progress in developing new, automated production and measuring instrument
has led to the 100% real-time inspection, where critical dimensions are measured and verified while
parts are being produced.The above mentioned quality is more critical in bio engineering field and
especially in implants. By this manufacturing cost reduces and further preventing processing of
2
defective parts along the manufacturing stages [1]. So, the present research has involved the quality
assessment in tissue engineering scaffolds during its fabrication and manufacturing stages and all the
artifacts of quality are being assessed [2, 3, 4]. The quality control can be done with the help of robotic
equipment to find the compliant and incompliant scaffolds for the total automated diagnosis of the part
mainly considering the pore size and other geometric features which are critical.
1.2 NETWORK BASED QUALITY ASSESSMENT SETUP
The equipment present at our lab for online inspection of the scaffolds manufactured using 3D
printer consists of mainly three robots, namely Yamaha YK 350X SCARA and Yamaha YK 180X
SCARA robots and a linear actuator robot which are being controlled using three robotic controllers two
RXC240 and one RCX 222 respectively and the parts are brought into the field of robot using a
conveyor system and sensor system. The parts are inspected through two Cognex micro smart vision
cameras. The overall robotic system used for quality control is as shown below:
Figure 1.1: Work Area set up at ISEL, UTEP
3
Here the scaffold models are placed on the conveyor and the program is written into the memory
of the robot using Yamaha language such that the sensors detect the moving part on the conveyor and it
stops, at the same time robot sends a signal to machine vision systems to take a picture of the part and
then it compares with the details in its memory and sends a signal back to robot either the scaffold to
continue on the conveyor or it will pick and drop the defective in a separate bin.The overall setting of
the network based system at Intelligent Systems Engineering Laboratory at UTEP is shown in figure
below.
Ethernet
Wireless Router
Robot and vision server Robot controllerSCARA Robot
Machine vision system
Web Camera
Client 1 Client 2 Client 3
USERS
Figure 1.2: Overall Setting of system at ISEL, UTEP
The operations on the robot are written using robotic code to the controller using Yamaha robotic
language which is connected to the PC using the onboard Ethernet card, which is an optional device for
connecting over the internet. Using TCP/IP internet protocols, a standard internet protocol can be used
to communicate with the controller. This unit uses 10BASE-T specifications and UTP (unshielded
4
twisted pair) or STP (shielded twisted pair) cables can be used. Computers are connected to the
controllers using telnet. Commands can be sent to controller when connection is established between
controller and PC. Application programming interface (API) is developed by embedding programming
code and input/output details using the visual basic code [5,6]. Figure 1.3 shows the screen shot of API.
The API is helpful in creating remote connection between PC and robot and other accessories integrated
with the robot. Web cameras were fixed in such positions that the remote operators can view the entire
process through them and make necessary changes to the program in the robot controller. The
connection between API and the controller was established by using Winsock components using various
ActiveX controls that communicate through IP addresses.
Figure 1.3: Application Programming Interface
5
1.3 RESEARCH OVERVIEW
The inspection of scaffolds is generally performed using scanning electron microscope and the
characterization process is manual. To avoid human intervention and common errors occurred during
characterization, the process can be performed with an automated setup using robotic system consisting
of conveyor and machine vision system. The main reason behind the interest for this kind of research is
due to the fact that usage of internet for automation is present trend which might have several effects as
well as there are several advantages with the online control of the systems. Here, when the whole
process is automated, using the communication module of the Cognex insight software, one can
remotely operate the robot and obtain the detailed data for the specific scaffold. To ensure quality
data, the outcomes from the present data will be used to train the neural networks and perform data
mining, so that the predictions from the data will be useful for future research. Data which is obtained
from the remote connection setup is analyzed using multi layer perceptron neural network and
radial basis function networks which are more useful and give accurate results in classification
and pattern recognition. Also, using design of experiments regression analysis is performed and
results from neural networks is compared with DOE.
1.4 THESIS ORGANIZATION
This thesis is organized into five chapters. Chapter1 gives introduction and thesis overview
respectively. Chapter 2 gives a formal literature review on the topics of tissue engineering scaffolds, 3D
printer, machine vision systems, neural networks. Chapter 3 includes the methodology of data analysis
using Design of Experiments and MLP & RBF neural networks respectively. Chapter 4 describes the
data mining conducted, its results and analysis. Finally, Chapter 5 concludes the thesis with a summary
of the research conducted and discussion of findings is presented along with the conclusions derived. In
addition, potential areas for further study are briefly discussed.
6
Chapter 2: Literature Review
2.1 QUALITY ASSESSMENT
There are many definitions of quality but the widely accepted definitions are fitness for use,
conformance to requirements, and the totality of characteristics of an entity that bear on its ability to
satisfy stated and implied need. An increase in quality not only reduces cost but also improves the
productivity which in turn reduces the rework and bad samples. Quality is very important in present
scenario, since it has moved beyond inspection and is gaining importance as strategic tool for improving
efficiency [7]. New manufacturing concepts like just in time, total quality management, flexible
manufacturing systems, rapid manufacturing has tremendous impact on improving productivity and in
turn the quality of the product. There are numerous research reports available on productivity and
quality improvements. Most of them deal with benefits that could be achieved by localized
improvements such as set-up reduction, smaller batch sizes, use of computers in information systems.
However, they do not offer any concrete strategic approach on how to improve productivity and quality
of the whole [8]. In this research we focus on improving the productivity and quality of a tissue
engineering scaffold using automated inspection. The objective of the work in this paper is to define:
• Identification of the source of quality defect on scaffold,
• Analysis of defect probability,
• Classification and regression analysis of quality,
• Predict the future probability of defect and improvements that can be inferred from tool
7
2.2 TISSUE ENGINEERING
2.2.1 INTRODUCTION
Bone is a polymer-‐inorganic nano composite at the macro level and mineralized collagen fibrils
arranged in the form of layers of interconnected pores at the micro structural level. An ideal implant
should have the same structure and similar composition as the bone. One of the approaches includes
production of implants using scaffolds. The shape and architecture of the scaffold are important as they
support the cells in proliferation, migration, differentiation and in producing extracellular matrix to
generate new tissues. The interconnected porosity of the scaffolds is necessary for vascularisation,
which enables cells to proliferate deeper into the scaffold. Rapid prototyping technologies enable us to
provide scaffolds with well defined and controlled internal architecture.
2.2.2 DESIGN REQUIREMENTS:
1. Mechanical requirements: Scaffold structures must possess the required mechanical stiffness
and strength of the replaced structure. This requirement helps the bone to bear load transfer using
implant and can be able to maintain its cell proliferation [9]. So using the correct biomaterial
required for implant with sufficient strength, stiffness and we also can adjust the porosity of the
scaffold to obtain the desired properties.
2. Geometrical requirements: It must be of a geometric size and shape that fits in at the site of
replacement. The defect site must be imaged and will be digitally reconstructed to aid in the
design of the scaffold external geometry. The geometry will be reconstructed into a CAD model
that can be used to instruct manufacturing systems for final fabrication in the required shape.
3. Manufacturing requirements:The selected biomaterial that satisfies the first requirement must
be compatible with the available manufacturing process to fabricate the scaffold. The selected
manufacturing process must also be capable to reproduce the intended design in terms of
8
required external and internal architectural features of the scaffold. Any post processing steps
that may be required should not damage the scaffold or should not leave chemical residues on the
final scaffold.
4. Biological requirements: The designed scaffold must facilitate cell attachment and distribution,
growth of regenerative tissue and facilitate the transport of nutrients, oxygen, chemical cues and
removal of wastes. This requirement can be achieved by controlling the porosity of the structure,
by providing pore interconnectivity inside the structure. Interactions between cells and
extracellular matrix are some of the key factors to study cell migration, proliferation,
differentiation, andapoptosis, which all are critical functions for tissue-engineered construct [9].
Table 2.1: Biometric design considerations and possible design solution
Design Considerations Possible design solution
Mechanical requirements
• Scaffold structural integrity
• Internal architectural stability
• Scaffold strength and stiffness
• Biomaterial selection
• Internal architecture
• Porosity and pore distribution
• Fabrication method
Geometrical requirements
• Anatomical fitting
• Scaffold external geometry
Manufacturing requirements
• Process ability
• Process effect
• Process controlled algorithms using
appropriate process planning
instructions
Biological requirements
• Cell loading, distribution and nutrition
• Cell attachment and in growth
• Cell-tissue aggregation and formation
• Biomaterial selection
• Preferred internal architecture and
layout
• Pore size and interconnectivity
• Vasculature
9
2.2.3 DESIGN OPTIMIZATION AND FABRICATION:
Design optimizing is the key step in scaffolds with proper shape and inner micro structure, which are
very important factors in tissue engineering. Scaffolds should have same shape as the defective bone, so
that the scaffolds can be placed well in body and guide bone’s growth correctly. So, the configurations
with the characteristics and properties such as porosity, shape, surface to volume ratio, pore size, pore
interconnectivity, shape (overall geometry), structural strength and bio compatibility. These factors are
the most critical factors in designing and fabricating a tissue engineering scaffolds. So the design
optimizing is an important step for obtaining scaffolds with proper shape and inner microstructure.
Traditionally fabrication methods like fiber bonding, solvent casting, particulate leaching, membrane
lamination, melt molding, gas forming, and cryogenic induced phase separation. However most of these
above stated techniques are based on manual work and there should an extra procedure associated with
this and getting suitable shape and microstructure is not that easy since they cannot be controlled well.
To overcome these limitations with the traditional fabrication techniques, such as rapid prototyping has
been explored by many scientists. Based on type of manufacturing these are further divided and
appropriate machines are used to produce the bio parts. The following table describes different types of
fabrication methods with description of each.
10
Table 2.2: Currently applied 3D fabrication technologies[10]
Fabrication technology Processing
Material properties
Scaffold design
Achievable pore size
Porosity in % Architecture
solvent casting in combination with particulate lamination casting soluble
user, material and technique sensitive 30-300 20-50
spherical pores, salt particles remain in matrix
Membrane lamination
solvent bonding soluble
user, material and technique sensitive 30-300 <85 irregular pore structure
fabrication of nonwoven
carding, needling, plate pressing fibers
machine controlled 20-100 <95
insufficient mechanical properties
melt molding molding Thermo plastics
machine controlled 50-500 <80
extrusion in combination with particulate leaching
extrusion with dies
Thermo plastics
machine controlled <100 <84
spherical pores, salt particles remain in matrix
emulsion freeze drying casting soluble
user, material and technique sensitive <200 <97
high volume of interconnected micro pore structure
thermally induced phase separation casting soluble
user, material and technique sensitive <200 <97
high volume of interconnected micro pore structure
supercritical fluid technology casting amorphous
material and technique sensitive <100 <30
high volume of interconnected micro pore structure
supercritical fluid technology in combination with particle leaching casting amorphous
material and technique sensitive <50 <97
low volume of non interconnected micro pore structure combined with interconnected macro pore structure
3D printing in and without particle leaching
solid freeform fabrication soluble
machine and computer controlled 45-150 <60
100% interconnected macro pore design and fabrication layer by layer, by use of water based binder incorporation of biological agents into matrix possible
fused deposition modeling
solid freeform fabrication thermoplastics
machine and computer controlled >150 <80
100% interconnected macro pore structure, design and fabrication layer by layer
11
Based on layer by layer manufacturing process, parts with complex shape or structure could be
produced through RP technologies easily and rapidly. Several kinds of RP technologies such as stereo
lithography (SLA), selective laser sintering (SLS), fused deposition modeling (FDM)[11], three-
dimensional printing (TDP or 3DP)[11] and so on, have been applied widely in fabricating bionic
scaffolds for tissue engineering and achieved some progress.Using technologies like 3 dimensional
printing (3DP) and selective laser sintering (SLS), can produce powder porous scaffolds for tissue
engineering. A key advantage of both technologies is that a large variety of materials can be used as
long as they are available in the form of powder. Dimensional accuracy is limited in these processes by
the nozzle size / laser diameter, control over the print head / laser movement and positioning, and the
particle size of the powder. 3D printing (3DP) is one prospective RP technique that may be used in
manufacturing hard tissues. 3D printings (3DP), Selective laser sintering (SLS) and fused deposition
modeling (FDM) are the most promising techniques that can be used in hard tissue manufacturing.
2.2.4 LITERATURE SURVEY FOR 3D PRINTING:
3D printing was invented at MIT and is being used to build TE scaffolds. Biological agents and other
materials that act as growth factors and biological agents can be incorporated into the process for
making the scaffold more effective. However in this method, resolution is limited by jet size which
makes it difficult to fabricate scaffolds with fine microstructures. The porosity of the scaffold prepared
using this technique is found to be low and there is also necessity for improvement in mechanical
properties of the scaffolds fabricated. 3D printing is simple method of fabricating models that works
similar to ink jet printing. In the process of fabrication, a liquid binder is ejected from the printer head
on to the layer of powder and the next layer of powder is stacked on to the existing layers of powder.
The function of the binder is to bind the powder particles between the layers. A limited number of
12
powder-‐binder combinations are available with most common being use of organic solvents like
chloroform as binder in fabrication of scaffolds.
Figure 2.1: 3D printing step by step process
Various biomaterials and fabrication techniques are being tested for bone tissue engineering for
more than a decade. Xiaohua Liu et al. [12] got published a review of polymer materials, scaffold design
and fabrication techniques investigated till that time. The paper also reviews the architectural parameters
of the scaffold. Though other factors like cell sources, regulating molecules, mechanical stimulation,
bioreactor design, cultivation conditions and clinical considerations are important for successful
development of a tissue, they are not discussed in this paper. Natural polymers though have the
advantage of biological recognition, but are limited with respect to control over their mechanical
properties and biodegradability. These facts provoked researches to try synthetic polymers. Rapid
prototyping is one of the few techniques employed for building bone tissue engineering scaffolds. The
main advantage of this technique is to be able to build parts directly from CAD model.
Uwe Gbureck et al. [13] attempted to manufacture custom-‐made calcium pyrophosphate implant
structures and scaffolds via 3D powder printing process using calcium phosphate cement setting
reaction. Samples were prepared using the TCP powder synthesized and diluted phosphoric acid with
13
different concentrations varying from 5% to 30%. Layer thickness of 100µm, binder-‐volume ratio of
0.28 for shell and 0.14 for core with a saturation level of 89% is adopted for printing. Surfaces of the
failed samples were examined using scanning electron microscope. Printing process allowed producing
components with a resolution of ±200µm. Scaffolds are three dimensional structures that promote tissue
formation.
HA is used to manufacture scaffolds as it is stoichiometrically similar to the inorganic part of the
bone. This study by Barbara Leukers et al. [14]focuses on the histological evaluation of the seeded
scaffolds, scaffolds being manufactured using 3D printing technology. A special scaffold was designed
to maximize the surface and facilitate the seeding process to enhance cell adhesion and supply of
nutrients. A spray dried HA granulate containing polymeric additives was used for manufacturing
scaffolds. A water soluble polymer blend (Schelofix) was used as binder. Powder based 3D printing
process was found to induce micro porosity which increases the surface accessibility of the scaffold for
fluid medium. From the above findings, it can be concluded that HA scaffolds made by 3D printing are
highly suitable for bone tissue engineering. HA and TCP are commonly used for making implants.
Alaadien Khalyfa et al. [15]developed a powder mixture comprising tetra calcium phosphate
(TTCP) as reactive component and β-‐tricalcium phosphate or calcium sulphate as biodegradable fillers.
The developed mixture could be useful in bone repair applications in load bearing areas. All the above
research has worked on the scaffold fabrication and characterization is done using sophisticated
microscope. The present research focuses on using machine vision systems for geometric analysis based
on which once if the program is written into the registry of the camera, by activating the suitable
program for the given dimensions, the whole characterization process can be completed in less than a
minute. But, efforts are being made to increase the capacity of the camera by extending the lens of the
camera with a microscope lens so that the images of size less than 500µm can be analyzed with ease.
14
2.3INTERNET BASED ROBOTICS
Internet based robotics is assembly planning system for intelligent task level programming in
SCARA robots. Web robots enable client to specify tasks, enable automated task/motion planning, and
generate robot control. Clients will communicate with the robot administrator using application
programming interface (API), a graphic user interface and send commands to the robot [17]. For low to
medium volume production needing frequent changes, robot based programmable systems have proved
to be advantageous to achieve successful automation. Three approaches exist for the programming of
industrial robots namely teach-in programming by robot guidance, Using explicit robot level computer
language and specifying task level sequence of states or operations. To implement web robotic
application we need web accessible production equipment, web-enabled monitoring system and web-
based decision functions [18]. The entities that are connected through web like the machine vision
camera which enables the quality control process are connected using LAN network using specific IP
address by which it can access over internet using its IP and Gateway addresses and port number.
Inspection, vision guidance and quality check can be performed remotely and instantly. The camera is
connected to the break out board which contains 8 input/output ports which can be helpful in
transmitting and receiving signals related to quality control (fail, pass and warning). Also, geometric and
visible features from the camera can be sent over through break out board. Sensors that are placed on the
conveyor on either side detect the part movement and send command to the machine vision system and
robot controller receives signal related to quality control [19, 20]. The Appendix -2(robot code) shows
the program code related to automated quality control using internet based robotics.
2.4 DATA MINING - NEURAL NETWORKS
The nervous system is inspiration to the artificial neural networks concept. Biologically speaking
neural networks are comprised of group of interconnected, functionally associated neurons. In general, a
15
neural netork consists of simple processing elements called neurons which exhibit complexbehavior,
based on their connections with the element and element parameters. These networks are extremely
complex. Hence, artificial neural networks are a mathematical model or computational model based on
biological neural network [23].
There are various learning models that are being used presently in the artificial intelligence
world. In this field artificial neural networks are proved worthy in applying to solve particular problems
such as image analysis, adaptive control, construction of intelligent software agents, speech recognition
pattern recognition and autonomous robots. On the other hand neural networks are used for physical and
mathematical modeling of neural systems in field of cognitive modeling. When used without
qualification, the terms “Neural Network” (NN) and “Artificial Neural Network” (ANN) usually refer to
a Multilayer Perceptron Network. However, there are many other types of neural networks including
Probabilistic Neural Networks, General Regression Neural Networks, Radial Basis Function Networks,
Cascade Correlation, Functional Link Networks, Kohonen networks, Gram-Charlier networks, Learning
Vector Quantization, Hebb networks, Adaline networks, Heteroassociative networks, Recurrent
Networks and Hybrid Networks. Here in this research we consider Multilayer perceptron network and
radial basis function which are more related to classification and pattern recognition. The following
figure shows a normal model of a neuron. I have included a review from latest researchers related to
neural networks and its applications.
∑
Input signals Synaptic weights
Summoning function
Bias
Activation function
Fig 2.2: Model of a neuron
16
Table 2.3: Review of Neural Networks related research
Author Method Case Study Inferences
Raphael Feraud,
Fabrice Clerot et.al
Multi layer
perceptron neural
networks
Defined performance
measure for interpreting
classifiers
The clustering algorithm and
graphical analysis allow neural
network to analyze without
constraints hence performance
has been preserved
Liang Chen, Wei
Xue, Naoyuki
Tokuda et.al
Multi layer
perceptron neural
networks
Accuracy of gender
classification and human
face recognition
Bi label classification problems
with un districted and districted
neural networks to divide
networks and test show better
results than large one
David Casasent,
Zue wen Chen et.al
Radial basis
function with
Neyman –Pearson
classification
Agricultural product
inspection
Database of 942 nuts shows
probability of false alarm
reduced to 30% for supervised
learning algorithm, and reduce
bad nuts in crops from 3 to 1%
Ganesh
Arulampam,
Abdesselam
Bouzerdoum et.al
Generalized feed
forward neural
network for
SIANN’s
3 bit parity, Pima Indians
Diabetes, Wisconsin
Breast Cancer problems
Breast cancer problem has error
under 1% . GFNN performed
well with more than 66%
networks giving correct
results.Diabetes problem has 768
samples with eight real valued
inputs and two output classes
17
where test error were as low as
17.7%
Markku Sermala,
Martti Juhola, Erna
Kentala et.al
Imbalanced class
distribution using
NetSet
Otoneurological data sets
with 38 variables
Used projection scaling approach
to compare simple approach to
classification results from NetSet
Nektarios A
Valous, Fernando
Mendoza, Da Wen
Sun, Paul Allen
et.al
Supervised MLP
neural network
Pork ham samples Using SVD for extracting visual
features like color, texture shape
and distribution of structures and
using MLP to classify ham
qualities with 86.1% accuracy.
Pawalai
Karaipeerapun,
Chun Che Fung
et.al
Binary
classification using
interval
neutrosphic sets
and neural
networks
Ionosphere, pima and
liver from UCI
Repository are used
Two approaches are created, one
with single pair of NN, second
using ensemble pair of NN with
each pair providing true and
false. Proposed ensemble
methodology provides better
classification results.
Lance E Besaw
Donna Rizzo et.al
Hierarchical
artificial neural
networks &counter
propagation
algorithm
VTANR (Vermont)
statewide assessment
data
Study shows two artificial neural
network algorithms to classify
stream sensitivity to create
consistent repeatability process
Sung Nien Yu, Independent ECG beat types from ICA is used to extract important
18
Kuan To Chou
et.al
component
analysis and neural
networks
MIT-BIH arrhythmia
database
features from ECG signals, these
features serve as input feature to
probabilistic neural networks and
back propagation neural
networks, both show
classification accuracy of over
98%
Taskin Kavzoglu Artificial neural
networks
Visualization samples of
geographical data in
Trabazon, black sea
region of Turkey
Quality and size of training
samples are important for
successful classification, neural
networks are powerful in
performance when operated with
expert knowledge
Yuksel Ozbay,
Gulay Tezel
Neural networks
with adaptive
activation function
ECG signals for 40 males
and 52 females
The idea of using neural
networks is novel idea in
biomedical data. NNAAF shows
an average accuracy rate for
models in training as 99.9% &
testing 98.2%.
Mehmet Korurek,
Berat Dogan
Radial Basis
function and
Particle swarm
optimization
ECG data with six
variables from MIT –
BIH arrhythmia data
base.
Results show that amount of
neurons required for K-NN is
more than proposed
methodology. The proposed
method can classify whole
19
training with half time required
for K-NN.
Wing W.Y.Ng
et.al
Radial basis
function with
minimization of
local error
Design of image
classifiers formed by low
primitives defined in
setting of MPEG-7
Intrusive experimentation show
resulting classifier outperforms
classifiers like SVM
Seongkuen Park,
Jae Pil Hwang,
Euntai Kim et.al
Multi layer
perceptron neural
networks using
probabilistic target
classification
Using 1853 Vehicle
samples and 2202
pedestrian samples for
24GHz microwave radar
sensor
Developed a target classification
for active safety system and
Classification performance is
greatly improved when MLP is
trained to classify target based on
radar outputs.
Emilson Pereria
Leite, Carlos
Roberto de Souze
Filho
Mat lab code for
neural networks
Generic textural images
& synthetic radar
aperture radar imagery
Mat lab code called TEXTNN
program is used to test, train and
analyze NN models for given
case study
Roland Linder,
Andreas E Albers
et.al
Artificial neural
networks
Set of 120 voice samples
were analyzed
Using prototype software
Approximation and classification
of medical data (ACMD ) data
was classified with accuracy of
80%, sensitivity of 63% and
specificity of 94%
20
2.4.1 BACKGROUND
The McCulloch and Pitts [40] publication is 1943 is base for the artificial neural networks
development. In their paper they created an idea about simple model of brain cells being used as neurons
and create network with those neurons. They called these neurons as MCP which are based on threshold
logic units, here a single neuron receives signal from environment and compares those signals using
threshold function for desired output. MCP neurons are binary in nature and activation function is used
to compare product of inputs and its weights to the output using binary functions. But those MCP
neurons had limitations. You could implement any Boolean function, but had to design each one. So
overcome the difficulties second generation of neural networks are being introduced. The basic network
architectures for these networks are shown below:
2.4.2 NETWORK ARCHITECTURES
2.4.2.1 SINGLE-LAYER FEED FORWARD NETWORKS
Single-layer feed forward networks are perceptrons. In a layered network the neurons are
organized in forms of layers. In single layer we have input layer which connects to output layer of
neurons. They are the simplest form of layered networks, consisting of an input layer of source nodes
that project onto an output layer of neurons (see Figure 2.3). These networks are strictly feed forward or
acyclic. We do not count the input layer neurons because these are independent (i.e., no computation
nodes).
21
Input 1
Input 2
Input 3
N1
N2
N3
Figure 2.3: Diagram of a single-layer feedforward artificial neural network.
2.4.2.2 NON LINEARFEED FORWARD NETWORKS
Multilayer Perceptrons
The second type of networks is the multi layer networks in which we have a layer of hidden
neurons. The function of hidden layers is to intervene between input and network output and help in
creating higher order statistics in other words the networks becomes global with an extra dimension of
interaction between perceptrons. The network is said to be fully connected only when each node of a
layer is connected to all nodes of the corresponding hidden layer or output layer. The figure 2.4 shows
the multi layer feed forward network connection. The multilayer perceptrons will propagate error term
backward through the feed forward network until the required objective function has been achieved.
This error propagation is performed using back propagation algorithm.
22
Input 1
Input 2
Input 3
H1
H2
H3
N1
N2
Figure 2.4: Diagram of a multi-layer feed forward artificial neural network.
Back propagation Algorithm
From the multi layer perceptrons using back propagation algorithm it has been most prominent
being creation of learning rule that could adjust weight between input layers called as the hidden layers.
Rumelhart, Hinton, and Williams [41] developed the popular back propagation rule. The back
propagation algorithm “consists of two passes through the layers of the perceptrons, a forward pass and
a backward pass”. Collection of inputs is put to the network and weights of the corresponding network
product is taken forward and using multiple hidden layers the output is forwarded towards the end of the
network and the observed output is compared to the desired output, where an error is generated and
propagated backwards into the network. This process is iterated until the required stopping criteria have
been achieved. The back propagation algorithm is a supervised learning method and is an
implementation of the delta rule (i.e. Least-Mean-Square Algorithm [42]) from gradient descent learning
(see Learning Algorithms). By presenting many inputs with the desired outputs and then applying back-
propagation of the error, the perceptron is "trained" to produce desired outputs with an increasing degree
of correctness [43]."
23
Radial Basis Function
A Radial Basis Function (RBF) neural network has an input layer, a hidden layer and an output
layer. The neurons in the hidden layer contain Gaussian transfer functions whose outputs are inversely
proportional to the distance from the center of the neuron. RBF networks are similar to K-Means
clustering and PNN/GRNN networks. The main difference is that PNN/GRNN networks have one
neuron for each point in the training file, whereas RBF networks have a variable number of neurons that
is usually much less than the number of training points [44].
An RBF network positions one or more RBF neurons in the space described by the predictor
variables. This space has as many dimensions as there are predictor variables. The Euclidean distance is
computed from the point being evaluated to the center of each neuron, and a radial basis function (RBF)
(also called a kernel function) is applied to the distance to compute the weight (influence) for each
neuron. The radial basis function is so named because the radius distance is the argument to the
function. Different types of radial basis functions could be used, but the most common is the Gaussian
function. The further a neuron is from the point being evaluated, the less influence it has.
Distance Activation
Distance
Figure 2.5: Figure showing RBF activation function and effect of distance on activation
24
If there is more than one predictor variable, then the RBF function has as many dimensions as
there are variables. The best predicted value for the new point is found by summing the output values of
the RBF functions multiplied by weights computed for each neuron. The radial basis function for a
neuron has a center and a radius (also called a spread). The radius may be different for each neuron, and,
in RBF networks generated by DTREG, the radius may be different in each dimension. With larger
spread, neurons at a distance from a point have a greater influence. Figure 2.6 shows effect of spread on
network.
Small spread, very selective
large spread, not very selective
Fig 2.6: Figure showing the effect of spread on the neuron
RBF Network Architecture
V 1
V 1
O 1
O 1
O 1
W 1W 2
W 3
Output
Input
∑
Fig 2.7: Radial Basis Function network model
25
RBF networks have three layers:
1. Input layer – There is one neuron in the input layer for each predictor variable. In the case of
categorical variables, N-1 neurons are used where N is the number of categories. The input
neurons (or processing before the input layer) standardize the range of the values by subtracting
the median and dividing by the interquartile range. The input neurons then feed the values to
each of the neurons in the hidden layer.
2. Hidden layer – This layer has a variable number of neurons (the optimal number is determined
by the training process). Each neuron consists of a radial basis function centered on a point with
as many dimensions as there are predictor variables. The spread (radius) of the RBF function
may be different for each dimension. The centers and spreads are determined by the training
process.
3. Summation layer – The value coming out of a neuron in the hidden layer is multiplied by a
weight associated with the neuron (W1, W2... Wn in this figure) and passed to the summation
which adds up the weighted values and presents this sum as the output of the network. For
classification problems, there is one output (and a separate set of weights and summation unit)
for each target category. The value output for a category is the probability that the case being
evaluated has that category.
Various methods have been used to train RBF networks. One approach first uses K-means clustering to
find cluster centers which are then used as the centers for the RBF functions. However, K-means
clustering is a computationally intensive procedure, and it often does not generate the optimal number of
centers. Another approach is to use a random subset of the training points as the centers. The
computation of the optimal weights between the neurons in the hidden layer and the summation layer is
26
done using ridge regression. An iterative procedure developed by Mark Orr [45] is used to compute the
optimal regularization Lambda parameter that minimizes generalized cross-validation (GCV) error.
Table 2.4: Comparison between Multi Layer Perceptrons and Radial Basis Function networks [23]
Term Multilayer Perceptron Radial Basis Function
Hidden layer MLP have one or more hidden
layers for each network
RBF in its basic form contains
single hidden layer also called as
hidden function.
Computation Nodes In MLP the hidden layer as well as
output layer has same neuron model
In RBF, the hidden layer
computation nodes are different
from output layer and serves
different purposes
Linearity In MLP, all the layers of network are
non linear when used as pattern
classifier
In RBF, the hidden layer is non
linear where as the output layer is
linear
Activation function The activation function is the
product of the input vector and the
synaptic weight vector
Here the activation function is the
Euclidean distance between input
vector and center of that unit
Approximations MLP construct global
approximations to non linear input-
output mapping, i.e., requires small
number of parameters than RBF of
same degree of accuracy
RBF construct local
approximations to exponentially
decaying localized non linearity’s
(Gaussian functions) to non linear
input output mappings.
27
2.4.3 APPLICATIONS
Artificial neural networks are usually applied to the following tasks: Function approximation, or
regression analysis, including time series prediction and modeling. Classification, including pattern and
sequence recognition, novelty detection and sequential decision making, Application areas include
pattern recognition (radar systems, face identification, object recognition, medical diagnosis(cancer
detection), sequence recognition (gesture, speech, handwritten text recognition), system identification
and control (vehicle control, process control), game-playing and decision making (backgammon, chess,
racing), financial applications (automated trading systems), data mining, visualization and e-mail spam
filtering.
In biomedical research, neural networks are often used for analysis and classification of an
experiment's outcomes. Traditional techniques include the linear discriminant function and the analysis
of covariance. Usually, the outcome of the experiment is dependent on a nonlinear function. Stubbs [46]
gives an overview of three biomedical applications using neural networks. One of the application areas
is drugdesign, where a three-layer back propagation neural network was developed to predict the
frequency of serious adverse reactions cases for 17 particular non-steroidal anti-inflammatory drugs,
using four inputs, each representing a particular property of the drugs. The developed neural network
model was able to predict the frequency of side effects of 17 different drugs with less than 5 percent
error. This network could be used to predict the adverse reactions rate for new drugs. The author
concludes that neural networks can be used for drug design and discovery and to provide information for
patient care.
28
2.5 DESIGN OF EXPERIMENTS
Design of experiments has become a highly developed area with a number of textbooks which
explain the backgrounds of the statistical methods. A short compilation of most useful designs for the
purpose of thermal spraying, based onto a literature review from this area will be discussed in this
section. The compilation will be categorized into two-level full factorial designs (2k), two-
level fractional factorial designs (2k − m) and response of surface methodology (RSM) designs. Design of
experiments deals with planning, conducting, analyzing and interpreting controlled tests to evaluate the
factors that control the value of a parameter or group of parameters [21]. A strategically planned and
executed experiment may provide a great deal of information about the effect on a response variable due
to one or more factors. A well-performed experiment may provide answers to questions such as what are
the key factors in the given process, what the optimal parameters for the model, main and interaction
effects in the process and parameters that could give less variation in output. Here, in case of scaffolds
five parameters on digitizing uncertainty, fractional factorial design is employed and considering three
replicates. Therefore the design is generator is given by the model I = ABCDE ensures all five factors
and factor interactions will not be aliased with each of themselves [22]. This follows the sequential
experimentation strategies to reduce the time and costs and to increase efficiency. This concludes all the
literature review related to the research topic.
29
Chapter 3: Methodology
This chapter is divided into four important sections namelyneural networks data mining
consisting of multi layer perceptron and radial basis function, a mathematical example for each showing
the neural networks performance and design of experiments for regression analysis. The methodology
for this research consists two important phases where phase one is classification analysis byneural
networks using Statistica 9 and comparing the results obtained from multilayer perceptron and radial
basis function networks and phase two being regression analysis of neural networks from multi layer
perceptron, radial basis function and design of experiments and comparing the error term obtained from
each of these models.
The scaffolds are fabricated using UGS NX6.0 modeling software and by varying pore size and
shape (circle and hexagon), were converted into stl(stereo lithography) files and were sent to 3D printer.
Using the Z Corp 3D printer using material Z Corp powder and Z Corp binder, the powder scaffolds are
manufactured which are useful in tissue engineering scaffolds for bone implants in general. The models
that are built are inspected on Cognex machine vision systems for measuring pore size, distance between
pores, number of pores, shape and surface area available and the data collected is analyzed using neural
networks and design of experiments concept for classification and regression analysis. The conceptual
framework for this methodology is presented in the figure below:
30
Scaffold samples fabrication and manufacture
Inspection using robotic and machine vision system facility
Identifying optimal values
Data acquisition
Data analysis for regression using neural networks
and DOE
Using regression analysis finding out the output porosity and comparing with
target porosity
Compute Error term associated with each network and give best method for the data
Using data for future predictions
Preparing Data for Classification analysis
using neural networks
Finding output predictions and comparing with
target
Finding the accuracy for each network type and selecting best
network
Figure3.1: Conceptual framework for methodology development
31
3.1 DESIGN SELECTION
Tissue-engineering techniques generally require the use of a porous, bioresorbable scaffold,
which serves as a three-dimensional (3D) template for initial cell attachment and subsequent tissue
formation, both in vitro and in vivo. Ideally, a scaffold should have the following characteristics:
1. A suitable macrostructure to promote cell proliferation and cell-specific matrix production.
2. An open-pore geometry with a highly porous surface and microstructure that enables cell in
growth.
3. Optimal pore size employed to encourage tissue regeneration and to avoid pore occlusion.
4. Suitable surface morphology and physiochemical properties to encourage intracellular signaling
and recruitment of cells.
Experimental observations reveal that the porosity of the scaffold built depends upon four main
parameters: slice thickness, road width, raster gap, raster angle. The slice thickness is the thickness of
the layer used to build the model layer by-layer; the road width is the width of the extruded layer; the
raster gap is the gap between the laying roads within a sliced plane of the part; and raster angle is the
angle between the succeeding horizontal raster layers of the model. Therefore, the experimental value of
the porosity P can be calculated by the equation:
⎥⎦
⎤⎢⎣
⎡⎟⎟⎠
⎞⎜⎜⎝
⎛−=
a
t
vvP 1 (3.1.1)
Where: !! = apparent volume (total volume) of the model, !! = true volume of model (volume
occupied by material)
We assume that large porosity for vascularization is of prime importance, as long as both
scaffold and regenerate tissue stiffness are maintained within an acceptable range, then the optimization
problem denoted as the porosity design can be written as
Objective function: ⎟⎟⎠
⎞⎜⎜⎝
⎛ −
t
a
E vv
scaffols
1max (3.1.2)
Where: !!"#$$%&' = Scaffold base material young’s modulus, !! = pore diameter
32
By varying the surface area, pore architecture and pore volume for the void volume of random,
porous architectures we proposed to describe the relationships between void volume components and the
structural and material properties as well as the dominant design characteristics governing the strength
of such porous architectures. The basic demand of tissue engineered scaffolds is that they be porous
enough to support interconnectivity which has been demonstrated to be around 60% porosity by volume.
We have considered two shapes of scaffold pores including the basic circular shape, and hexagon was
considered whose surface area and volume are being calculated and significant data was collected for
the above equation. Therefore, the factors that affect the design of scaffold are shape, size, number of
pores, raster gap, and surface area. The data obtained was analyzed using design of experiments and
then neural networks for data mining and predictions for future models.
3.2 NEURAL NETWORK MODEL
Here we use two types of networks namely: multilayer perceptron and radial basis function. These
two neural networks are only used because these are much superior to others in classification
analysis for this kind of data.
3.2.1 MLP MODEL
The terms that are used in this multi layer perceptron models are
!! = output units,
!! = input units
!!" = weights
g ( ) = activation functions
! ! = error term in network
p = number of training patterns
33
M = number of output units
ɳ = the learning rate between (0,1)
∝ = momentum constant
Each hidden or output unit is called as a perceptron, and is a function of product of input vector and
weight associated with it.
⎟⎟⎠
⎞⎜⎜⎝
⎛+= ∑
=
n
jijiji bxwfY
1
(3.3.1)
Here we use gradient descent method to find out the error function to find the correct weights. The
errors are local to each node and change in weight from node I to output j, !!" is controlled by the
input travels along the connection and error signal from output j.
( ) iiiji xytw −=Δ (3.3.2)
Using the gradient descent method the error term is propagated back through the model. The
algorithm which helps in performing this is called as back propagation algorithm. It has two passes
namely the forward pass and backward pass. The forward pass computes functional signal and helps
propagate input patterns through the network. Backward pass computes error signal and propagates
the error backwards through network starting at output units. Suppose we have three layered
network,
))(()()()( tugtxtvgtz ij
jiji =⎟⎟⎠
⎞⎜⎜⎝
⎛= ∑ (3.3.3)
))(()()()( tagtztwgty ij
jiji =⎟⎟⎠
⎞⎜⎜⎝
⎛= ∑ (3.3.4)
Where a, u are activations for the activation function g ( ) at time t.
In general sigmoid (logistic) is used as activation function in MLP.
34
Therefore,
)(11
))(exp(11))(( tka
ii ietkatag
−+=
−+= (3.3.5)
Derivative of sigmoid function is given by, ))(1)(())((' tytkytag iii −=
Where, k is a positive constant. The sigmoid function gives value in range of [0, 1].
During the forward pass the values of hidden and outputs units respectively are as follows:
))(()()(
))((
)()()(
tagyztwta
tugztxtvtu
kk
jkjk
ij
jijj
=
=
=
=
∑
∑
During the backward pass the error signal is propagated forwards. And in general, we use the normal
error term which is sum of squares which is given by
∑=
−=1
2))()((21)(k
kk tytdtE , Where !! is the target value for dimension k.
We use gradient descent method for modifying weights for both hidden units and output units, i.e.,
(3.3.6)
From partial derivation of the above term we get,
)()(
)()(
)()(
twta
tatE
twtE
ij
i
ijij ∂
∂
∂
∂=
∂
∂ (3.3.7)
Where first part of right hand side is to determine error for pattern changes for network from input I
to hidden j. The second part is to determine how the net input to unit j changes as function of change
in weight w.
)()()()1(twtEtwtw
ijijij ∂
∂−−+ α
35
)()()(;
)()()(,
)()()(
),()()(
tatEt
tutEtThen
tztwtutx
twta
ii
ii
jij
ij
ij
i
∂
∂−=Δ
∂
∂−=
=∂
∂=
∂
∂
δ
Therefore, the hidden units are given by:
∑∑ Δ−=∂∂
∂∂=
∂
∂−=
jjji
ji
ii
i
ii wtag
tutatatE
tutEt ))(('
)()()()(
)()()(δ
Therefore, the output units are given by:
( ))()())((')()())(('
)()()( tytdtag
tytEtag
tatEt kki
ii
ii −−=
∂
∂−=
∂
∂−=Δ
(3.3.8)
Therefore, the weight terms given by
)()()()(
)()()()(
tzttwtE
txttvtE
jiij
jiij
Δ=∂
∂−
=∂
∂− δ
So achieve gradient descent in E should change weight. The weight transformation functions for
hidden units and output units respectively are given by
)()()()1( txttvtv jiijij ηδ=−+
)()()()1( tzttwtw jiijij Δ=−+ η (3.3.9)
Where, ɳ is the learning rate between (0,1) (0 < ɳ ≤ 1)
The algorithm is repeated until stopping criterion has been achieved.
Stopping criterion for the network is given by,
( )∑∑= =
−=p
i
M
jkk tytdE
1 1
2)()( (3.3.10)
Where, p = number of training patterns, M = number of output units.
36
We can stop training when rate of change of E is small, suggesting there is convergence in data.
Network is trained using one of two following techniques: Sequential mode (on-line, stochastic, or
per-pattern) in which weights updated after each pattern is presented and Batch mode (off-line or
per -epoch) in which we calculate the derivatives/weight changes for each pattern in the training set
and then calculate total change by summing individual changes. Using momentum term we can
reduce the instability when rate of convergence increases. Therefore, the weight term change is
observed as follows:
[ ])1()()()()()1( −−+=−+ twtwtyttwtw ijijjiijij αηδ (3.3.11)
Where ∝ is momentum constant and controls how much notice is taken of recent history. Using this
momentum term, weight changes tend to have same sign momentum terms increases and gradient
decrease speed up convergence on shallow gradient. Also, If weight changes tend have opposing
signs momentum term decreases and gradient descent slows to reduce oscillations (stabilizes).
37
Example
Here, I explain a small XOR example for multi layer perceptron. Considering two inputs [0,1] and
two outputs [1,0] for multi layer perceptron for training. Using learning rate ɳ = 0.1and calculate
activations for 1st layer
1110011 =+×+×−=u
2111002 =+×+×=u
= -‐1
= 0
= 0
= 1
= 1
= -‐1
= 0
= 1
= 1
= 2
Therefore, the first layer outputs through functions(from equation 3.3.3) are
2)( 22 == ugZ
= -‐1
= 0
= 0
= 1
= 1
= -‐1
= 0
= 1
= 1
= 2
Using second layer outputs in similar way using equation 3.3.4,
211 == ay
222 == ay
1)( 11 == ugZ
38
Using backward pass for target [1, 0] using equation 3.3.8, so !!=1 and !!=0. So,
( ) 1111 −=−=Δ yd
( ) 2222 =−=Δ yd
= -‐1
= 0
= 0
= 1
= 1
= -‐1
= 0
= 1
= 1
= 2
= -‐1
= -‐2
= -‐2
= -‐4
And !! =1 &!! = -2 are used to calculate weight changes, the weights are again propagated in
forward direction again until the required target is reached.
= -‐1
= 0
= 0
= 1
= -‐ 1
= 2= 0
= -‐2
= -‐1
= -‐2
Here we can see that from network diagram that target has been close enough to required target. So,
we can stop the process. So, the required target [1, 0] is achieved by [ !! = 1.66,!! = 0.66 ]
= -‐1
= 0
= 0.1
= 0.8
= 0.9
= -‐1.2
= 0.2
= 0.6
= 1.66
= 0.32
39
3.2.2 RBF MODEL
The variables that are used in this multi layer perceptron models are
! = network weights
! = input feature vector
! = number of hidden units
!! = associated value of desired output
∅! ! = Gaussian activation function
!! , ! = mean and covariance of matrix
In radial basis function, the Euclidean distance is computed from the point being evaluated to the center
of each neuron, and a radial basis function (RBF) (also called a kernel function) is applied to the
distance to compute the weight (influence) for each neuron.
Weight = RBF (distance) (3.3.12)
Various functions are used as activation functions for RBF networks, but for pattern classification
Gaussian function is preferred over others. The Gaussian activation function for RBF network is given
by:
⎥⎦
⎤⎢⎣
⎡−−−= ∑
−1
)()(exp)(j
jT
jj XXX µµφ , for j=1…L (3.3.13)
Where X is input feature vector, L is number of hidden units, !! and ! are the mean and covariance
of the matrix of the !!! Gaussian function. Statistically, an activation function models a probability
density function where !! and ! represent first and second order statistics. The output layer is a
weighted sum of hidden unit outputs:
40
∑=
=L
jkjkk XX
1)()( ϕλψ ,for k = 1…M
[ ])(exp11)(
XXY
kk ψ−+
= , for k = 1…M (3.3.14)
Where, !!" are the output weights, each corresponding to the connection between hidden unit and output
unit and M represent number of output units. Using the generalized radial basis function network, we
take the condition number of matrix as ratio of largest Eigen value to the smallest Eigen value of the
matrix. Haykin states that reduction of complexity of network is necessary to overcome computational
difficulties. The approximation procedure is used to find out the sub optimal solution from Galerkin’s
method in variation problems. According to this technique approximation solution on finite basis is:
F*(X) = ∑=
M
iii Xw
1
)(ϕ (3.3.15)
Where )(Xiϕ , i=1,2…M is new set of basis functions which are assumed to be linearly dependant.
Considering radial basis functions, Haykin gives the following equation:
( )itxGX −=)(ϕ i=1,2…M (3.3.16)
Therefore, the equation 3.3.15, can be rewritten as
F*(X) = ( ) ( )iN
ii
N
iii txGwtXGw −=∑∑
== 11
; (3.3.17)
Then Haykin says that equationcan be rephrased as squared Euclidean norm
2GWd − (3.3.18)
41
Where,
[ ]Ndddd ,...., 21= .
From this equation GW=d, we can find out the desired output and weights corresponding to network
where G is matrix of green functions and W is matrix of weight vectors
Example
Consider the same XOR problem in radial basis function. Here we consider all four points on the plane
(1,1), (1,0) , (0,1) and (0,0). So we need to have pattern classifier with binary input 0 for patterns (1,1)
and (0,0) and binary input 1 for patterns (0,1) and (1,0). Using Gaussian hidden activation functions,
[ ]Ttx te 1,1, 111 == −−φ ,
[ ]Ttx te 0,0, 122 == −−φ
Therefore, the hidden functions for XOR problem are:
Input pattern (X) First hidden function ∅! (X) First hidden function ∅! (X)
(1,1) 1 0.1353
(0,1) 0.3678 0.3678
(0,0) 0.1353 1
(1,0) 0.3678 0.3678
42
The decision making diagram for the radial basis function is
(1,1)
(0,0)
(0,1)(1,0)
Decision bundary
The relationship between input and output of network is given by
jj dxy =)( , j = 1,2,3,4
Where !!input is vector and !! is associated value of desired output. Using the Euclidian norm equation
3.3.18, GW=d, the outputs of the hidden units corresponding to four patterns, using the equation below,
( )ijji txg −= , j=1, 2, 3, 4; i =1, 2
14241132311222111211
gggggggg
13678.03678.0111353.013678.03678.011353.01
=G =
! = 1 0 1 0 !
! = ! ! ! !
43
Therefore, the input output transformation can be computed for XOR problem as:
Data point, j Input pattern (X) Desired output, !! Actual output, !!
1 (1,1) 1 0.901
2 (0,1) 0 -0.01
3 (0,0) 1 0.901
4 (1,0) 0 -0.01
Using !!G, the weight matrices are calculated as
692.1284.2284.2
−
W =
Therefore, the RBF network solving XOR problem is given by,
W
W
Input nodes Gaussian functions Linear output neuron
Fixed input = -1
b(bias)
44
3.3 DESIGN OF EXPERIMENTS
From the design of required scaffolds, we have the objective function is to maximize porosity for
given material E (young’s modulus). The parameters that are of prime importance are shape, size,
number of pores, raster gap and surface area of the scaffold. Using, the concepts of design of
experiments we perform a fractional factorial design for the given case, considering half factorial
design, possible numbers of experiments are 2!!!and we take replications to be 4 to make sure that
the experiment is performed to the required accuracy. The total number of scaffolds that needed to
be analyzed is 16 x 4 = 64. Therefore, the level of interest for all the five factors is shown below for
a low high range. This research assumes that higher order interaction between factors is negligible.
The order of 64 experiments is randomized first. Then these experiments are conducted on
inspection station using machine vision system and robot setup.
Table 3.1: Factors and levels of interest for Design of Experiments
Level
Factors
Shape Surface Area Radius Distance No. of Pores
Low (-1) Circular 0.785 mm 0.5 mm 2 mm 60
High(+1) Hexagonal 3.14 mm 1 mm 4 mm 120
These runs are performed in Minitab statistical software, ANOVA is performed to get the effect of
factors on the model, Also regression analysis in conducted on the model. The response optimizer
from the design of experiments module gives the optimal value of each factor, so that the optimal
design will be achieved for given set of operating conditions. The design for the whole experiment is
shown in the table below:
45
Table 3.2: Design of the model with data for Design of Experiments
Radius Distance No. of pores Shape Surface Area
Porosity
1 2 3 4
-1 -1 -1 -1 1 0.1687 0.191 0.1598 0.1721
1 -1 -1 -1 -1 0.5314 0.6016 0.418 0.5792
-1 1 -1 -1 -1 0.429 0.5866 0.299 0.418
1 1 -1 -1 1 0.682 0.6823 0.4704 0.495
-1 -1 1 -1 -1 0.3925 0.3403 0.377 0.453
1 -1 1 -1 1 0.6598 0.5866 0.6193 0.6436
-1 1 1 -1 1 0.3622 0.5196 0.3638 0.377
1 1 1 -1 -1 0.7524 0.7628 0.705 0.7109
-1 -1 -1 1 -1 0.1844 0.257 0.199 0.1565
1 -1 -1 1 1 0.6908 0.6891 0.7056 0.7568
-1 1 -1 1 1 0.1795 0.1795 0.2242 0.2248
1 1 -1 1 -1 0.7328 0.6891 0.6937 0.8053
-1 -1 1 1 1 0.605 0.6427 0.641 0.383
1 -1 1 1 -1 0.7627 0.7281 0.7248 0.7141
-1 1 1 1 -1 0.6213 0.6056 0.6411 0.6455
1 1 1 1 1 0.7876 0.7546 0.7632 0.788
46
Chapter 4: Case study
This chapter discusses about the case study related to sample data collection for scaffolds. First, the
drawing of the scaffold part is modeled using modeling software UGS NX 6.0, several models of
scaffold are drawn with varying size(0.5-1mm) and shape(circle, hexagonal). The modeled diagrams are
converted into stereo lithography files.In the following chapters we will address quality issues related to
fabrication and manufacture of tissue engineering scaffolds using 3D Printer. These were then converted
to STL file and sent to 3D Printerfor fabrication, and then these models are characterized at the
inspection station using Cognex Machine Vision Systems which have the capability of comparing
present dimensions to the specified dimensions. The following figure shows screenshot of scaffold
model in UGS NX6.0
Figure4.1: Screen shot of scaffold model in UGS NX 6.0
47
The following figure 3.2 shows the step by step process for complete case study with fabrication
using Auto CAD, manufacturing using 3D printer and investigation using machine vision system.
Figure 4.2: Step by step process for complete fabrication, manufacture and inspection
Robotic facility is used to complete the automation process
Using insight software of machine vision system to complete characterization
Using de-powdering from 3D printer to complete scaffold sample
Using 3D Printer Z 450 to create scaffold samples
Z Print is used to upload models to 3D printer
Creating models for circular and hexagonal shapes
Modeling of scaffold using UGS NX 6.0
48
These files are imported to the Z Print printing software for Z450 3D printer and all different
samples are put on the printer bed for production. These are then sent to the printer Z450; the printer
is connected to the computer by Ethernet. Samples are produced using Z corporation powder and Z
corporation binder as the material. The parts that are built dried for 1 hour and then we have to de-
powder (vacuum) to remove powder from the voids. The final part is brushed to remove powder on
the surface, and then glued to make sure that the scaffold is not fragile. This procedure is repeated
for all the samples. The following figure shows working on 3D printer Z450.
Figure 4.3: (a) working with Z corporation 3D printer Z450 (b) Using Z Print for setting up models
for printer. (c) 3D printer making prints of scaffold models.
49
The following picture shows the hexagonal scaffold and circular scaffold that are being modeled
using Auto CAD.
Figure 4.4: Screen shot of hexagonal and circular scaffolds
These are then put on the inspection station consisting of robot, conveyor and machine vision
system. The dimensions of sample are analyzed using machine vision system. In machine vision
system, the system is first calibrated to scale either millimeters or inches. Then, the sample under
investigation is trained to the system and coming samples dimensions are measured and stored and
are measured relative to the first, then the machine vision system gives back signal to the robot either
to keep sample moving or move to another bin. Once, the program is running in robot controller and
samples are on the conveyor, we can judge sample good or bad based on its pore diameter, pore
architecture and raster gap. The robot program is written in Y- language to automate the whole
50
inspection process. Appendix -2 shows the robot program codethat has been used to integrate the
conveyor, robot and machine vision system facilities. The following figure shows sample
investigation under machine vision system.
Figure 4.5: Screen shot of Cognex insight explorer with scaffold being investigated.
51
A total of 134 samples are collected which have either hexagonal shaped or circular shaped pores.
The scaffolds are produced from 0.5-1mm with increment of 0.1mm; therefore there are six cases of
each. The Appendix -1 shows the experimental data for scaffolds. The following figures show the
variation in size of scaffold pore for hexagonal and circular shape.
Figure 4.6: Hexagonal scaffolds with varying pore size from 0.5 -1mm.
Figure 4.7: Circular scaffolds with varying pore size from 0.5 -1mm.
52
Chapter 5: Analysis
Analysis of data obtained is performed by two methods:classification analysis and regression
analysis. The classification analysis consists of analysis of neural networks namely multi layer
perceptron and radial basis function and regression analysis is performed using neural networks (multi
layer perceptron and radial basis function) and design of experiments For performing analysis with
neural networks, commercially available software Statistica 9.1, for design of experiments Minitab 15
are used.
5.1 CLASSIFICATION ANALYSIS
The data obtained from the case study explained in the previous chapter was analyzed with
various network ranges without making any modifications to the data. The statistica automated neural
network (SANN) module can be selected under data mining tab and by using classification analysis and
by automated neuron search (ANS) methodology the network has been trained.
5.1.1 MULTI LAYER PERCEPTRONS
The data was analyzed using three fold testing and validation. Software enables user to give the
network minimum and maximum and weight decay for the hidden neurons and output neurons. Also,
user can specify the mode of analysis train, test or validation by specifying the sample data column in
the spread sheet. A range of networks from 3 to 50 has been analyzed, the following table shows the
results obtained. Overall accuracy of 100% has been achieved with training. Also, there are instances of
no error in test and validation. The figure 5.1 shows screen shot of working with Statistica 9.1 for multi
layer perceptron and figure 5.2 shows screen shot of results window in MLP network.
53
Figure 5.1: Screen shot of multi layer perceptron working with Statistica 9.1
Figure 5.2: Result from multi layer perceptron neural network
54
The following table shows the network, test train and validation accuracy for multi-layer
perceptron network.
Table 5.1: Multi layer perceptron network analysis
Network Train Test validate
MLP 3 100 98.68 100
MLP 6 100 100 86.67
MLP 11 100 100 100
MLP 12 100 100 100
MLP 13 100 100 100
MLP 16 91.34 98.68 91.34
MLP 21 100 98.68 94.78
MLP 22 100 98.78 98.78
MLP 25 100 100 97.36
MLP 26 100 96.05 90
MLP 29 100 98.68 100
MLP 31 100 94.78 86.67
MLP 35 100 100 100
MLP 36 100 98.68 100
MLP 41 100 98.68 100
MLP 46 100 97.36 100
Using the networks above, the test train and validation predictions for the classification based on
good/bad are obtained. The figure 5.3 shows graph of MLP network analysis, figure 5.4 shows the
histogram of train, test and validation data and figure 5.5 shows result for networks in range 10-15.
55
Figure 5.3: Graph showing the MLP network error in each phase of analysis
Figure 5.4: Histogram of network type versus accuracy for MLP network
MLP 3
MLP 6
MLP 11
MLP 12
MLP 13
MLP 16
MLP 21
MLP 22
MLP 25
MLP 26
MLP 29
MLP 31
MLP 35
MLP 36
MLP 41
MLP 46
Train 100 100 100 100 100 91.34 100 100 100 100 100 100 100 100 100 100
Test 98.68 100 100 100 100 98.68 98.68 98.78 100 96.05 98.68 94.78 100 98.68 98.68 97.36
validate 100 86.67 100 100 100 91.34 94.78 98.78 97.36 90 100 86.67 100 100 100 100
84
86
88
90
92
94
96
98
100
Acc
urac
y
Multi layer perceptron network analysis
84
86
88
90
92
94
96
98
100
MLP 3
MLP 6
MLP 11
MLP 12
MLP 13
MLP 16
MLP 21
MLP 22
MLP 25
MLP 26
MLP 29
MLP 31
MLP 35
MLP 36
MLP 41
MLP 46
Acc
urac
y
Network type
Multi layer perceptron network train, test & validate histogram
Train
Test
validate
56
Figure 5.5: Result from multi layer perceptron neural network for networks 10, 13 & 14
The table shows results of target and output decisions for predictions based on ensemble predictions
Table 5.2: Target and output decisions for MLP based on ensemble predictions
Decision Target Decision output - ensemble Decision residuals ensemble
Good Good Correct
Good Good Correct
bad bad Correct
bad bad Correct
bad bad Correct
bad bad Correct
bad bad Correct
bad bad Correct
Good Good Correct
Good Good Correct
Good Good Correct
bad bad Correct
Good Good Correct
Good Good Correct
57
bad bad Correct
bad bad Correct
Good Good Correct
Good Good Correct
Good Good Correct
Good Good Correct
bad bad Correct
bad bad Correct
Good Good Correct
Good Good Correct
Good Good Correct
Good Good Correct
Good Good Correct
bad bad Correct
bad bad Correct
5.1.2 RADIAL BASIS FUNCTION
The data was analyzed using three fold testing and validation. Software enables user to give the
network minimum and maximum and weight decay only for output neurons. Also, user can specify the
mode of analysis train, test or validation by specifying the sample data column in the spread sheet. A
range of networks from 14 to 50 has been analyzed, the following table shows the results obtained.
Overall accuracy of 100% has been achieved with training. Also, there are instances of no error in test
and validation. The figure 5.1 shows screen shot of working with Statistica 9.1 for radial basis function
neural networks.
58
Figure 5.6: Result from radial basis function neural network
The following table shows the network, test train and validation accuracy for radial basis
function network. Table 5.3: Radial basis function neural network analysis
Network Train Test Validate
RBF 14 96.55 96.05 96.67
RBF 20 100 93.42 96.67
RBF 21 100 97.36 93.33
RBF 25 100 100 96.67
RBF 26 100 100 96.67
RBF 30 100 96.05 96.67
RBF 35 100 96.05 93.33
RBF 40 100 96.05 90
RBF 41 100 98.68 93.33
RBF 45 100 100 96.67
59
Using the networks above, the test train and validation predictions for the classification based on
good/bad are obtained. The figure 5.7 shows graph of RBF network analysis, figure 5.8 shows the
histogram of train, test and validation data.
Figure 5.7: Graph showing the RBF network error in each phase of analysis
Figure 5.8: Histogram of network type versus accuracy for RBF network
RBF 14
RBF 20
RBF 21
RBF 25
RBF 26
RBF 30
RBF 35
RBF 40
RBF 41
RBF 45
Train 96.55 100 100 100 100 100 100 100 100 100
Test 96.05 93.42 97.36 100 100 96.05 96.05 96.05 98.68 100
Validate 96.67 96.67 93.33 96.67 96.67 96.67 93.33 90 93.33 96.67
84 86 88 90 92 94 96 98
100 102
Acc
urac
y
Radial basis function network analysis
84 86 88 90 92 94 96 98
100
RBF 14 RBF 20 RBF 21 RBF 25 RBF 26 RBF 30 RBF 35 RBF 40 RBF 41 RBF 45
Acc
urac
y
Network type
Radial basis function network train, test & validate histogram
Train
Test
Validate
60
The following figure shows the screenshot of ensemble output predictions for classification analysis
with all inputs and outputs included.
Figure 5.9: Screen shot showing target and output for decision with ensemble predictions
The table shows results of target and output decisions for predictions based on ensemble predictions
Table 5.4: Target and output decisions for RBF based on ensemble predictions
Decision Target Decision output - ensemble Decision residuals ensemble
Good Good Correct
Good Good Correct
bad bad Correct
bad bad Correct
Good Good Correct
Good Good Correct
bad Good Incorrect
61
bad bad Correct
Good Good Correct
Good Good Correct
bad bad Correct
bad bad Correct
bad bad Correct
Good bad Incorrect
Good Good Correct
bad bad Correct
bad bad Correct
Good Good Correct
Good Good Correct
Good Good Correct
Good Good Correct
bad bad Correct
bad bad Correct
bad bad Correct
Good Good Correct
Good Good Correct
Good Good Correct
Good Good Correct
bad bad Correct
bad bad Correct
62
5.1.3 ANALYSIS OF RESULTS
Using the data from Appendix 1, and making decision (good/bad) as the categorical target and
the factors in analysis like radius, shape, distance, surface area, porosity and number of pores as
continuous inputs the analysis has been performed for multi layer perceptron and radial basis function
neural networks. It was found that both the multi layer perceptron and radial basis function work well
with this data and has resulted in highest classification accuracy. According to thumb rule, the hidden
number of layers should be 10% of whole data set, i.e., hidden layers for artificial neural networks
should be around 11-15. This heuristic has been supported with the results from the analysis. For MLP,
the classification accuracy is 100% for train, test and validation data for networks in the range 11-15. So,
from table 5.1, we can see that the classification accuracy in case of MLP classification accuracy is
maximum for MLP 11, MLP 12, and MLP 13. Similarly, in case of radial basis function neural network
the maximum classification accuracy is (100,100, 96.67) for train, test and validation respectively. In
case of RBF, the network could not analyze the data completely and give validation accurately. The
predictions for the test data are analyzed to find out the wrong predictions. STATISTICA provides
option to see the predictions for all the cases tested. Figure 5.9 shows the screen shot of the same. Also,
from table 5.4, we can observe that for validation data set we have 2 incorrect ensemble predictions for
the maximum classification accuracy for networks RBF 25, RBF 26, and RBF 45.
63
5.2 REGRESSION ANALYSIS
Regression analysis estimates the conditional expectation of dependant variable for the given
independent variables; it is widely used for forecasting and prediction. So, for the given case study
regression analysis is performed to find out the effect of each independent variable on the dependable
variable, porosity. The following figure shows the porosity distribution for all the samples in the case
study. In this analysis we compare the neural networks regression model with design of experiments
design model and find out the error term for each model and perform comparison.
Figure 5.10: Porosity distribution for the given data
5.2.1 MULTILAYER PERCEPTRON NEURAL NETWORK
Using the multi-layer perceptron neural network, we define the network model similar to the
case of classification analysis, here instead of the whole sensitivity analysis process, the best network
chosen from the classification analysis is considered for the regression model. So, dividing the data into
three fold, where two thirds of the data is considered train and remaining as test and validation out of
which 50% is chosen as validation and other 50% for testing. The resultant prediction from the neural
0
0.2
0.4
0.6
0.8
1
0 10 20 30 40 50 60 70
Perc
enta
ge
Samples
Porosity Distribution
Sample porosity
64
network model for the output porosity is compared with the target porosity and error term is generated
using the model. The following figure shows the screen shot of the window showing multi-layer
perceptron network analysis results and graph showing the porosity target and porosity output
predictions ensemble.
Figure 5.11: Screen shot showing result window for MLP network.
Figure 5.12: Graph of Samples versus porosity performing regression analysis for MLP network
0.150000
0.250000
0.350000
0.450000
0.550000
0.650000
0.750000
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61
Poro
sity
Sample
Regression Error Analysis for MLP Neural network
Porosity Target
Porosity output-‐ensemble
65
The table shows the results for sample type, target porosity and output porosity ensemble of all
the porosity outputs in each MLP network.
Table 5.5: Multi layer perceptron regression analysis predictions
Multi layer perceptron regression predictions
Sample Porosity Target Porosity output ensemble
Validation 0.168700 0.168417
Train 0.531400 0.532115
Train 0.429000 0.428725
Train 0.682000 0.682303
Train 0.392500 0.392750
Train 0.659800 0.660397
Train 0.362200 0.362084
Train 0.752400 0.750334
Test 0.184400 0.185173
Train 0.690800 0.690149
Train 0.179500 0.179488
Train 0.732800 0.732848
Test 0.605000 0.605486
Train 0.762700 0.763045
Validation 0.621300 0.621405
Test 0.787600 0.786977
Train 0.191000 0.191194
Test 0.601600 0.603600
66
Train 0.586600 0.586858
Train 0.682300 0.682303
Train 0.340300 0.340054
Train 0.586600 0.580938
Train 0.519600 0.519467
Train 0.762800 0.765358
Validation 0.257000 0.256642
Train 0.689100 0.689034
Validation 0.179500 0.179488
Train 0.689100 0.690149
Train 0.642700 0.642784
Train 0.728100 0.728597
Train 0.605600 0.605486
Train 0.754600 0.754314
Test 0.159800 0.159209
Test 0.418000 0.417516
Train 0.299000 0.299504
Train 0.470400 0.471188
Validation 0.377000 0.376997
Validation 0.619300 0.619580
Train 0.363800 0.363424
Train 0.705000 0.704574
Train 0.199000 0.198901
67
Train 0.705600 0.705494
Train 0.224200 0.224570
Train 0.693700 0.694195
Test 0.641000 0.641188
Train 0.724800 0.725315
Train 0.641100 0.641188
Train 0.763200 0.762752
Validation 0.172100 0.171798
Train 0.579200 0.580938
Train 0.418000 0.417516
Train 0.495000 0.495641
Validation 0.453000 0.453743
Train 0.643600 0.644181
Train 0.377000 0.376997
Test 0.710900 0.710071
Train 0.156500 0.156143
Train 0.756800 0.756335
Train 0.224800 0.224570
Train 0.805300 0.804624
Train 0.383000 0.382752
Validation 0.714100 0.714418
Test 0.645500 0.645605
Train 0.788000 0.787284
68
5.2.2 RADIAL BASIS FUNCTION NEURAL NETWORK
In the similar way, the regression analysis is performed with RBF network and target porosity is
compared with the ensemble output porosity and error term is generated through the network. The
following figure shows the screen shot of the window showing radial basis function network analysis
results and graph showing the porosity target and porosity output predictions ensemble.
Figure 5.13: Screen shot showing result window for RBF network.
Figure 5.14: Graph of Samples versus porosity performing regression analysis for MLP network
0.150000 0.250000 0.350000 0.450000 0.550000 0.650000 0.750000
1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61
Poro
sity
Sample
Regression Error Analysis for RBF Neural network
Porosity Target
Porosity output-‐ensemble
69
The table shows the results for sample type, target porosity and output porosity ensemble of all
the porosity outputs in each RBF network.
Table 5.6: Radial basis function regression analysis predictions
Radial basis function regression predictions
Sample Porosity Target Porosity output ensemble
Validation 0.168700 0.265192
Train 0.531400 0.539243
Train 0.429000 0.462485
Train 0.682000 0.717888
Train 0.392500 0.418979
Train 0.659800 0.632647
Train 0.362200 0.369603
Train 0.752400 0.738655
Test 0.184400 0.214087
Train 0.690800 0.662026
Train 0.179500 0.192152
Train 0.732800 0.689931
Test 0.605000 0.605999
Train 0.762700 0.721637
Validation 0.621300 0.643657
Test 0.787600 0.780054
Train 0.191000 0.265095
Test 0.601600 0.590646
70
Train 0.586600 0.578034
Train 0.682300 0.717888
Train 0.340300 0.344059
Train 0.586600 0.579656
Train 0.519600 0.527127
Train 0.762800 0.731348
Validation 0.257000 0.224096
Train 0.689100 0.669509
Validation 0.179500 0.192152
Train 0.689100 0.662026
Train 0.642700 0.653702
Train 0.728100 0.727813
Train 0.605600 0.605999
Train 0.754600 0.780958
Test 0.159800 0.266119
Test 0.418000 0.457484
Train 0.299000 0.313011
Train 0.470400 0.494669
Validation 0.377000 0.361180
Validation 0.619300 0.600138
Train 0.363800 0.354459
Train 0.705000 0.722541
Train 0.199000 0.255799
71
Train 0.705600 0.680901
Train 0.224200 0.231614
Train 0.693700 0.717566
Test 0.641000 0.651148
Train 0.724800 0.727602
Train 0.641100 0.651148
Train 0.763200 0.780297
Validation 0.172100 0.264254
Train 0.579200 0.579656
Train 0.418000 0.457484
Train 0.495000 0.494635
Validation 0.453000 0.478934
Train 0.643600 0.623470
Train 0.377000 0.361180
Test 0.710900 0.727115
Train 0.156500 0.203679
Train 0.756800 0.694571
Train 0.224800 0.231614
Train 0.805300 0.712231
Train 0.383000 0.350884
Validation 0.714100 0.715740
Test 0.645500 0.683082
Train 0.788000 0.742999
72
5.2.3 DESIGN OF EXPERIMENTS
To establish the prediction model, regression model and to find the impact of significant factors a
software package called Minitab 15 is used to perform ANOVA and regression analysis using the
experimental data from Appendix 1. Table 4.1 shows the working project on Minitab 15. Among the
five factors considered size, shape, distance, number of pores and surface area this analysis shows the
effect of each factor independently and interaction between factors on the model with a factor of
significance of 0.05.
Figure 5.15: Screen shot of Minitab 15 working on Design of Experiments
73
5.2.3.1 FACTORIAL FIT
Using fractional factorial design from design of experiments module, we have performed half
fractional factorial design on the model and taking the replications of each corner to be four, the design
has been created. The figure 4.2 shows the estimated effects and coefficients for porosity, we can also
observe from the ANOVA analysis that the effects of all the factors on porosity are significant and
interactions are significant as well. The figure 4.2 shows that this model has a satisfactory goodness of
fit for a factor of significance 0.05.
Figure 5.16: ANOVA result from Minitab
74
The graphs of normal probability plot, constant variance plot, histogram plot and plot versus fits
depicts that the data is normally distributed and validates that this model has shown satisfactory
goodness of fit for the given data.
0.20.10.0-0.1-0.2
99.9
99
9590
80706050403020
10
5
1
0.1
Residual
Perc
ent
Normal Probability Plot(response is Porosity)
Figure 5.17: Normal probability plot
Radius Distance No. of pores Shape Surface Area
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1
-1
1-1
1-1
1-1
1-1
1-1
1-1
1-1
1-1
1-1
-11
-11
1-1
-11
1-1
1-1
-11
1.21.00.80.60.40.20.095% Bonferroni Confidence Intervals for StDevs
Test Statistic 38.28P-Value 0.001
Test Statistic 1.43P-Value 0.174
Bartlett's Test
Levene's Test
Test for Equal Variances for Porosity
Figure 5.18: Test for equal variances
75
0.160.080.00-0.08-0.16
25
20
15
10
5
0
Residual
Frequency
Histogram(response is Porosity)
Figure 5.19: Histogram of residuals versus frequency
0.80.70.60.50.40.30.2
0.2
0.1
0.0
-0.1
-0.2
Fitted Value
Res
idua
l
Versus Fits(response is Porosity)
Figure 5.20: Residuals versus fits
76
5.2.3.2 RESPONSE SURFACE REGRESSION ANALYSIS
Using the response surface regression analysis from design of experiments module we can see
that the factors radius, distance, surface area, shape, number of pores have significant effect on porosity,
also we can see that the interaction between radius and number of pores, radius and shape, distance and
surface area, distance and shape, number of pores and shape, shape and surface area have significant
effect on the model. The response surface regression analysis is shown in figure 4.7 below.
Figure 5.21: Response surface regression analysis
77
Therefore, the regression model is given by,
Porosity(Y) = 0.52635 + 0.15136*radius +0.03465*distance between pores + 0.08097* number
of pores + 0.04166*shape -0.02108*surface area – 0.0422*(radius*number of
pores) + 0.01727*(radius*shape) – 0.01917*(distance between pores*shape) –
0.04906*(distance*surface area) + 0.02654*(number of pores*shape) +
0.01653*(shape*surface area)
5.2.3.3 RESPONSE OPTIMIZER
Using the response optimizer from regression analysis for design of experiments, from the figure
4.8 we can see the response optimizer for maximizing porosity. We can observe that maximum porosity
is obtained when radius, distance between pores, number of pores, and shape is high.
CurHigh
Low0.00000D
Optimal
d = 0.00000
MaximumPorosity
y = 0.8710
0.00000DesirabilityComposite
-1.0
1.0
-1.0
1.0
-1.0
1.0
-1.0
1.0
-1.0
1.0Distance No. of p Shape Surface Radius
[1.0] [1.0] [1.0] [1.0] [-1.0]
Figure 5.22: Response optimizer for regression analysis
78
The following figure shows graph of porosity target and porosity output predictions ensemble.
Figure 5.23: Graph of Samples versus porosity performing regression analysis for DOE
5.2.4 ANALYSIS OF RESULTS
From the results obtained using the regression analysis performed by three methods namely
multi layer perceptron neural networks, radial basis function neural networks and design of experiments.
Using the data from Appendix 1, and porosity as the categorical target and the factors in analysis like
radius, shape, distance, surface area and number of pores as continuous inputs the analysis has been
performed for multi layer perceptron and radial basis function neural networks. We could see that the
error term is lowest using multi layer perceptron neural networks, these networks are build so strong that
the testing accuracy, training accuracy and validation accuracy are almost perfect. The radial basis
function results are comparable with the design of experiments results, which show slight more variation
when compared to RBF network. These network analyses are performed using commercial software
called Statistica. The design of experiments regression analysis is performed using Minitab. The error
0
0.2
0.4
0.6
0.8
1
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64
Poro
sity
Sample
Regression analysis using Design of Experiments
Observed Porosity
Porosity from regression equaKon
79
term is lowest in multi layer perceptron and the value of error term is 0.000001. The following table
shows comparison of regression analysis for each of the three methods with the mean value of the
porosity and absolute error term associated with each method.
Table 5.7: Computational results for the regression analysis
R D No. S SA Mean DOE MLP RBF Error
-1 -1 -1 -1 1 0.1729 0.14375 0.172655 0.265165 0.02915 0.000245 0.092265
1 -1 -1 -1 -1 0.53255 0.8537 0.533542 0.541757 0.32115 0.000992 0.009207
-1 1 -1 -1 -1 0.43315 0.8464 0.433151 0.452754 0.41325 0.000001 0.019604
1 1 -1 -1 1 0.58243 0.7594 0.585859 0.606270 0.17698 0.003434 0.023845
-1 -1 1 -1 -1 0.3907 0.855 0.390886 0.400788 0.4643 0.000186 0.010088
1 -1 1 -1 1 0.62733 0.674 0.626274 0.608978 0.04668 0.001051 0.018347
-1 1 1 -1 1 0.40565 0.7296 0.405493 0.403092 0.32395 0.000157 0.002558
1 1 1 -1 -1 0.73278 0.7667 0.732584 0.729915 0.03393 0.000191 0.002860
-1 -1 -1 1 -1 0.19923 0.479 0.199215 0.224415 0.27978 0.000010 0.025190
1 -1 -1 1 1 0.71058 0.6898 0.710253 0.676752 0.02078 0.000322 0.033823
-1 1 -1 1 1 0.202 0.422 0.202029 0.211883 0.22 0.000029 0.009883
1 1 -1 1 -1 0.73023 0.7412 0.730454 0.695438 0.01098 0.000229 0.034787
-1 -1 1 1 1 0.56793 0.8181 0.568053 0.565433 0.25018 0.000128 0.002492
1 -1 1 1 -1 0.73243 0.8175 0.732844 0.723198 0.08507 0.000419 0.009227
-1 1 1 1 -1 0.62838 0.9217 0.628421 0.645971 0.29333 0.000046 0.017596
1 1 1 1 1 0.77335 0.7219 0.772832 0.771077 0.05145 0.000518 0.002273
80
Chapter 6: Conclusions
This research features the following contributions. First, the parameters involved in fabrication of
scaffold are considered simultaneously in this research. Second, the classification analysis is performed
for parameters under consideration to obtain the categorical target of good/bad for the scaffolds
prepared. Also, non-linear regression model and multi layer perceptron neural network involving back
propagation algorithm are used to develop models and estimate uncertainty. Fourth, the relative
comparison of error term predicted by these models is compared with experimental model using design
of experiments. This Study was conducted under the following assumptions:
1. The pore architecture of artificial sample is same as the original scaffold i.e., the material
properties are same
2. The measurement of radius of pore is representative of the entire pores in given surface area
3. The pore size measured by machine vision system is accurate
4. The machining errors during manufacturing are neglected.
The purpose of this work is to improve the quality involved behind manufacturing and using
scaffolds. Here, I have just considered the geometrical aspect of scaffold and used an artificial scaffold
model which has been fabricated and manufacturing using rapid manufacturing. Using the best
methodology, that gives out superior classification accuracy and regression accuracy at high speed.
Neural networks is selected because of its advantages both in classification analysis and regression
analysis for this specific case study, from the literature review out of many types of neural networks
because of classification analysis results from previous works multi layer perceptron and radial basis
function are selected for this research. Hence, two new approaches were designed and tested using
81
commercial software called Statistica. From the results and analysis in previous chapter it has been
proven that multi layer perceptron neural networks which has 100% classification accuracy. Also, radial
basis function which is another type of artificial neural networkshas shown the classification accuracy of
96.67%.
From the results, it can be concluded that the proposed method fared better compared to the other
because of improved classification accuracy. Also, the level of factor significance and regression
analysis was compared using Design of experiments (Minitab 15) with neural networks consisting of
multi layer perceptron and radial basis function. All the five factors that are used in this design are
randomized and results from chapter 5 prove the following. Neural networks perform better regression
analysis when compared to design of experiments and multi layer perceptron has the lowest error term in
predicting the porosity for the given model.Also, the results from DOE prove the factor of significance
for each independent factor on the whole model, this can be observed from response optimizer which
shows that radius of pore should be high, number of pores should be high, hexagonal shape is better and
distance between pores should be more as well. The measurements for hexagonal shape are measured
using inscribed circle for the hexagon. The results prove the literature that hexagonal shape is better than
circular pore because of its more edges and can be more helpful in cell culturing.
82
6.1 FUTURE WORK
Due to the above assumptions, the scaffold model could only be inspected for geometric and
manufacturing features. The model developed in this study exhibited following aspects that need to be
improved so that the inspection process is powerful and useful
1. This study is limited only to the geometrical and manufacturing features related quality
assessment of tissue engineering scaffolds. Also, one can perform the mechanical properties
testing using an analysis software like ANSYS for testing strength, hardness and other
mechanical features. Also, material chemical properties and possible interactions between
chemicals is also important.
2. Also, the biological requirements for scaffold to be used and cell culturing should also be
performed to see any possible corrections in the design of scaffold to create perfect implant.
3. The machine vision system could only inspect the top face of the sample, which restricts the
geometric inspection to 2 dimensions. There is a possibility of creating an graphical user
interface (GUI) for machine vision systems in which multiple machine vision systems are placed
around the sample to take the images of sample from top view, front view and side views so that
the GUI software can create the 3 dimensional image of the sample and inspection process can
also investigate the cross-sectional features and interaction between pores.
83
Bibliography
1. D, Garvin. "What Does “Product Quality” Really Mean?" 1984.
2. Kwon, Yongjin and Bill Tseng. "Fuzzy neuron adaptive modeling to predict surface
roughness under process variations in CNC turning." Journal of manufacturing systems
(2002): Vol.21, No.6, 440-450.
3. Fung, Chang-Xue and Xianfeng Wang. "Digitizing uncertainity modeling for reverse
engineering applications: regression versus neural networks." Journal of intelligent
manufacturing (2002): Vol 13, No.3,189-199.
4. Feng, Chang Xue. "Surface roughness predictive modeling: neural networks versus
regression." IIE transactions on design and manufacturing (2002).
5. Yamaha robot operational manual RCX 240.
6. Cognex smart image sensor frame work manual.
7. Dhafr, Nasreddin, et al. "Improvements of quality, performance in manufacturing
organizations by minimization of production defects." Robotics and Computer Integrated
Manufacturing (2006): 536-542.
8. A, Gunasekaran, et al. "Improving productivity and quality in manufacturing." International
journal of production economics (1994): 169-183.
9. Ma, Peter and Jennifer Elisseeff. Scaffolding in tissue engineering. CRC Press, 2006.
10. Hutmacher, Dietmar. "Scaffolss in tissue engineering bone and cartilage." Biomaterials
(2000): 2529-2543.
11. I, Zein, et al. "Fused deposition modeling of novel scaffold archtectures for tissue
engineering applications." Biomaterials (2002): 1169-1185.
12. Liu, Xiaohua and Peter X. MA. " Polymeric scaffolds for Bone tissue engineering. ." Annals
of Biomedical engineering (2004): 477-486.
84
13. Gbureck, Uwe. "Preparation of tricalcuim phospate/calcium pyrophosphate structures via
rapdi prototyping." Journal of material science:materials in medicine (2008): 1559-1563.
14. Leukers, Barbara. "Hydrozyapetite scaffolds for bone tissue engineering made by 3D
printing." Journal of material science:materials in medicine (2005): 1121-1124.
15. Khalyfa, Alladien. "Development of a new calcuim phospate powder binder system for 3D
printing of patient specific implants." Journal of material science:materials in medicine
(2007): 909-916.
16. JK, Sherwood, et al. "A three dimensional osteochondral composite scaffold for articular
cartilage repair." Biomaterials (2002): 4739-4751.
17. V.B, Sunil and Pande S.S. "WebRobot: Internet based robotic, assembly planning system."
Computers in industry (2004): 191-207.
18. Chiou, Richard, Prathaban Mookiah and Yongjun Kwon. "Manufacturing E-Quality throught
integrated web-enabled computer vision and robotics." International Journal of Advanced
Manufacturing Technology (2008): Vol.43, 720-730.
19. Kwon, Yongjin, Richard Chiou and Leonard Stepanskiy. "Remote condition based
maintenance for web enabled robotic system." Robotics and Computer-Integrated
Manufatcuring (2009): 552-559.
20. Y, Kwon, et al. "Remote control of quality using ethernet vision and web-enabled robotic
system." Journal of concurrent engineering (2006): 14-35.
21. Pierlot, Christel, et al. "Design of experiments in thermal spraying: A review." Surface and
coatings technology (2008): 4483-4490.
22. Feng, Xhang Xue Jack, et al. "Design and analysis of experiments in CMM measuring
uncertainity study." Precision Engineering (2007): 94-101.
85
23. Haykin, Simon. Neural Networks: A Comprehensive Foundation. Macmillan College
Publishing, 1994.
24. Feraud, Raphael and Fabrice Clerot. "A methodology to explain neural network
classification." Neural networks (2001): 237-246.
25. Chen, Liang, Wei Xue and Naoyuki Tokuda. "Classification of 2 dimensional arrya patterns:
Assembling many small neural networks is better than using a large one." Neural networks
(2010): 770-781.
26. Casasent, David and Xue-wen chen. "Radial basis function neural network for non-linear
fisher discrimination and neyman-pearson classification." Neural networks (2003): 529-535.
27. Arulampalam, Ganesh and Abdesselam Bouzerdoum. "A generalized feed forward neural
network architecture for classification and regression." Neural networks (2003): 561-568.
28. Siermala, Markku, Martti Juhola and Erna Kentala. "Neural network classification of
otoneurological data and its visualization." Computers in biology and medicine (2008): 858-
866.
29. Valous A, Nektarios, et al. "Supervised neural network classification of pre-sliced pork ham
images using quaternionic singular values." Meat science (2010): 422-430.
30. Kraipeerapun, Pawalai and Chun Che Fung. "Binary classifcation using ensemble neural
networks and interval neutrosophic sets." Neurocomputing (2009): 2845-2856.
31. Besaw, Lance, et al. "Stream classification using hierarchical artificial neural networks: A
fluvial hazard management tool." Journal of hydrology (2009): 34-43.
32. Yu, Sung-Nien and Kuan-To Chou. "Integration of independant component analysis and
neural networks for ECG beat classification." Expert system with applications (2008): 2841-
2846.
86
33. Kavzoglu, Taskin. "Increasing the accuracy of neural network classification using refined
training data." Environmental modelling and software (2009): 850-858.
34. Ozbay, Yuksel and Gulay Tezel. "A new method for classificationof ECG arrhythmias uisng
neural networks with adaptive actiavtion function." Digital signal processing (2010): 1040-
1049.
35. Korurek, Mehmet and Berat Dogan. "ECG beat classification using particle swarm
optimization and radial basis function neural network." Expert system with applications
(2010): 7563-7569.
36. W.Y.Ng, Wing, et al. "Image classifcation with the use of radial basis function neural
networks and the minimization of the localized generalization error." Pattern recognition
(2007): 19-32.
37. Park, Seongkeun, et al. "A neural network approach to target classifcation for active safety
system using microwave radar." Expert system with applications (2010): 2340-3246.
38. Leite, Emilson Pereira and Carlos Roberto de Souza Filho. "TEXTNN- A matlab program for
textural classiifcation using neural networks." Computers and Geosciences (2009): 2084-
2094.
39. Linder, Roland, et al. "Artificial neural network based classification to screen for dysphonia
using psychoacoustic scaling of acoustic voice features." Journal of voice (2006): Vol22,
No.2,155-163.
40. Pitts, W.H and W.S McCulloch. "A logical calculus of the ideas immanent in nevous
activity." Bulletin of mathematical biophysics (1943): 115-133.
41. D.E, Rumelhart, Hinton G.E and Williams R.J. "Learning internal represenations by error
propagation, parallel distributed processing: Explorations in microstructure of cognition."
MIT Press (1986): 318-362.
87
42. B, Windrow and Hoff Jr M.E. "Adaptive witching circuits." IRE WESCON convention
record (1960): 96-104.
43. M, Marsalli. Mc culloch-pitts neurons. 2010.
44. J, Park and Sandberg J.W. "Universal approximations using radial basis functions network."
Neural computation (1991): Vol.3, 246-257.
45. Orr, Mark, et al. "Combining regression trees and radial basis fucntions ." (n.d.).
46. D.F, Stubbs. "Three applications of neurocomputing in biomedical research."
Neurocomputing (1990): 61-66.
47. F, Bennis, Castagliola P and Pino L. "statistical analysis of geometric tolerances: A case
study." Journal of Quality Engineering (2005.): vol 17,no:3,419-427.
48. Data Mining, Statistical Analysis, Software and Services, Credit Scoring | StatSoft . 01 Nov
2010. <http://www.statsoft.com/textbook/stneunet.html>.
49. Narayan, Roger. Biomedical Materials. Springer, 2009.
88
Appendix
APPENDIX 1: EXPERIMENTAL DATA
Radius Distance No. of pores Surface Area Porosity Shape Decision Train/Test data
0.9195 4.12 94 2.656 0.7064 1 Good Train
0.9215 3.89 94 2.668 0.7095 1 Good Train
0.9415 4.56 94 2.785 0.7406 1 Good Train
0.8815 4.54 94 2.441 0.6492 1 Good Test
0.9235 4.97 94 2.679 0.7126 1 Good Validate
0.911 4.14 102 2.6076 0.7524 1 Good Train
0.867 4.05 102 2.3618 0.6815 1 Good Train
0.855 4.4 102 2.2968 0.6627 1 Good Train
0.9205 4.02 102 2.6622 0.7682 1 Good Test
0.903 4.3 102 2.562 0.7393 1 Good Validate
0.9085 4.23 93 2.5933 0.6823 1 Good Train
0.9035 4.07 94 2.5648 0.6820 1 Good Train
0.9015 2.9 61 2.5535 0.4406 1 bad Train
0.8685 2.54 69 2.3699 0.4626 1 bad Test
0.8645 2.87 80 2.34826 0.5314 1 bad Validate
0.877 2.5 88 2.4166 0.6016 1 Good Train
0.8955 2.83 70 2.5196 0.4989 1 bad Train
0.8705 2.71 84 2.3809 0.5658 1 bad Train
0.8825 3.05 62 2.4470 0.4292 1 bad Test
0.8005 2.42 92 2.0133 0.5240 1 bad Validate
89
0.8005 2.71 103 2.013 0.5866 1 bad Train
0.8105 2.1 113 2.0640 0.6598 1 Good Train
0.822 2.08 106 2.1229 0.6366 1 Good Train
0.775 2.42 76 1.8871 0.4057 1 bad Test
0.845 2.72 109 2.2434 0.6918 1 Good Validate
0.6005 2.42 113 1.1330 0.3622 1 bad Train
0.62 2.35 83 1.2077 0.2836 1 bad Train
0.617 2.69 116 1.1961 0.3925 1 bad Train
0.6435 2.4 71 1.3010 0.2613 1 bad Test
0.6785 2.23 127 1.4464 0.5196 1 bad Validate
0.64 2.1 107 1.2869 0.3895 1 bad Train
0.6975 2.12 81 1.5286 0.3502 1 bad Train
0.51 2.09 73 0.8172 0.1687 1 bad Train
0.575 1.95 65 1.0388 0.1910 1 bad Test
0.6 1.87 64 1.1311 0.2048 1 bad Validate
0.59 2.3 110 1.0937 0.3403 1 bad Train
0.525 2.14 50 0.8660 0.1225 1 bad Train
0.9485 4.12 94 2.8267 0.7517 1 Good Train
0.87 3.89 94 2.3781 0.6324 1 Good Test
0.9655 4.56 94 2.9289 0.7788 1 Good Validate
0.8675 4.54 94 2.3645 0.6288 1 Good Train
0.9735 4.97 94 2.977 0.7918 1 Good Train
0.8825 4.14 102 2.447 0.7061 1 Good Train
90
0.8855 4.05 102 2.4636 0.7109 1 Good Test
0.885 4.4 102 2.4608 0.7101 1 Good Validate
0.882 4.02 102 2.4442 0.7053 1 Good Train
0.969 4.3 102 2.9502 0.8513 1 Good Train
0.968 4.23 93 2.9441 0.7746 1 Good Train
0.9625 4.07 94 2.9107 0.7740 1 Good Test
0.9315 2.9 61 2.7262 0.4704 1 bad Validate
0.8825 2.54 69 2.4470 0.4776 1 bad Train
0.866 2.87 80 2.3563 0.5333 1 bad Train
0.8605 2.5 88 2.3265 0.5792 1 bad Train
0.892 2.83 70 2.4999 0.4950 1 bad Test
0.8625 2.71 84 2.3373 0.5554 1 bad Validate
0.871 3.05 62 2.38362 0.4180 1 bad Train
0.82 2.42 92 2.1126 0.5498 1 bad Train
0.8225 2.71 103 2.1255 0.6193 1 Good Train
0.8005 2.1 113 2.0133 0.6436 1 Good Test
0.8375 2.08 106 2.2038 0.6608 1 Good Validate
0.8355 2.42 76 2.1933 0.4715 1 bad Train
0.821 2.72 109 2.1178 0.6530 1 Good Train
0.649 2.42 113 1.3234 0.4230 1 bad Train
0.6375 2.35 83 1.2769 0.2998 1 bad Test
0.6325 2.69 116 1.2569 0.4125 1 bad Validate
0.6005 2.4 71 1.1335 0.2275 1 bad Train
91
0.6335 2.23 127 1.2609 0.4530 1 bad Train
0.63 2.1 107 1.2470 0.3774 1 bad Train
0.6475 2.12 81 1.3173 0.3018 1 bad Test
0.515 2.09 73 0.8333 0.1721 1 bad Validate
0.5255 1.95 65 0.86766 0.1595 1 bad Train
0.53 1.87 64 0.8825 0.1598 1 bad Train
0.61 2.3 110 1.1691 0.3638 1 bad Train
0.575 2.14 50 1.0388 0.1469 1 bad Test
0.9765 3.16 82 2.9960 0.6950 2 Good Validate
1.018 3.44 75 3.2561 0.6908 2 Good Train
0.9735 3.19 87 2.9776 0.7328 2 Good Train
0.974 3.48 86 2.9807 0.7252 2 Good Train
0.9665 3.24 83 2.9350 0.6891 2 Good Test
0.8785 2.89 110 2.4248 0.7546 2 Good Validate
0.8875 2.54 104 2.4748 0.7281 2 Good Train
0.8955 2.45 107 2.5196 0.7627 2 Good Train
0.8935 2.91 111 2.5083 0.7876 2 Good Train
0.892 2.9 97 2.4999 0.6860 2 Good Test
0.8815 2.54 112 2.4414 0.7735 2 Good Validate
0.86 2.51 120 2.3238 0.7889 2 Good Train
0.9045 2.59 123 2.5705 0.8944 2 Good Train
0.876 2.64 122 2.41102 0.8321 2 Good Train
0.893 2.58 121 2.5055 0.8577 2 Good Test
92
0.889 2.75 121 2.4831 0.8500 2 Good Validate
0.808 2.56 105 2.0512 0.6093 2 Good Train
0.8045 2.77 108 2.0335 0.6213 2 Good Train
0.8035 2.83 112 2.0285 0.6427 2 Good Train
0.798 2.53 107 2.0008 0.6056 2 Good Test
0.675 2.5 70 1.4315 0.2835 2 bad Validate
0.6475 2.1 116 1.3173 0.4322 2 bad Train
0.627 2.1 54 1.2352 0.1887 2 bad Train
0.578 2.03 71 1.0496 0.2108 2 bad Train
0.5765 2.14 84 1.0442 0.2481 2 bad Test
0.54 2.18 62 0.9162 0.1607 2 bad Validate
0.5565 2.09 58 0.9730 0.1596 2 bad Train
0.545 2.25 70 0.9332 0.1848 2 bad Train
0.5975 2.05 81 1.1217 0.2570 2 bad Train
0.5225 2.15 76 0.8577 0.1844 2 bad Test
0.9725 3.16 82 2.9715 0.6893 2 Good Validate
1.0465 3.44 75 3.4409 0.7301 2 Good Train
1.0205 3.19 87 3.2721 0.8053 2 Good Train
0.995 3.48 86 3.1106 0.7568 2 Good Train
0.978 3.24 83 3.0052 0.7056 2 Good Test
0.898 2.89 110 2.5337 0.7884 2 Good Validate
0.8855 2.54 104 2.4636 0.7248 2 Good Train
0.8665 2.45 107 2.3590 0.7141 2 Good Train
93
0.8795 2.91 111 2.4304 0.7632 2 Good Train
0.897 2.9 97 2.5280 0.6937 2 Good Test
0.8615 2.54 112 2.3319 0.7388 2 Good Validate
0.8615 2.51 120 2.3319 0.7916 2 Good Train
0.868 2.59 123 2.3672 0.8237 2 Good Train
0.91 2.64 122 2.6018 0.8980 2 Good Train
0.878 2.58 121 2.4221 0.8291 2 Good Test
0.916 2.75 121 2.6363 0.9024 2 Good Validate
0.836 2.56 105 2.1959 0.6523 2 Good Train
0.82 2.77 108 2.1126 0.6455 2 Good Train
0.8025 2.83 112 2.0234 0.6411 2 Good Train
0.821 2.53 107 2.1178 0.6410 2 Good Test
0.7225 2.5 70 1.6401 0.3248 2 bad Validate
0.6095 2.1 116 1.1672 0.3830 2 bad Train
0.5205 2.08 65 0.8512 0.1565 2 bad Train
0.645 2.1 54 1.3071 0.1996 2 bad Train
0.578 2.03 71 1.0496 0.2108 2 bad Test
0.548 2.14 84 0.9435 0.2242 2 bad Validate
0.565 2.18 62 1.003 0.1759 2 bad Train
0.58 2.09 58 1.0569 0.1734 2 bad Train
0.505 2.25 70 0.80125 0.1586 2 bad Train
0.575 2.05 81 1.0388 0.2380 2 bad Test
0.595 2.15 76 1.1123 0.2391 2 bad Validate
94
APPENDIX 2: ROBOT PROGRAM
ACCEL 80 /*SETTING ACCELERATION TO 80% OF TOTAL AVAILABLE*/
DECEL 80
SPEED 60
MOVE P, P0, Z=0 /*MOVING TO P0, INITIAL START UP POINT*/
*MAIN:
DO2(5)=0 /*SETTING THE TRIGGER OF CAMERA TO OFF*/
DELAY 5000
DO2(6)=1 /*SETTING THE CONVEYER TO ON*/
DELAY 1600
WAIT DI(30)=0/*WAITING FOR THE SENSOR TO DETECT THE PART*/
DO2(6)=0 /* SETTING THE CONVEYER TO STOP */
DELAY 50
DO2(5)=1 /*TRIGGERING THE CAMERA*/
DELAY 10000
IF DI(37)=0 THEN /*CHECKING IF THE PART IS BAD (DI(37) IS THE OUTPUT FROM
CAMERA)*/
MOVE P, P106, Z=0 /*MOVE TO THE POINT OVER THE BAD PART ON CONVEYER*/
DO2(0)=1 /*TRIGGERING THE SUCTION TIP 1 TO BE ON*/
DO2(1)=1/*TRIGGERING SUCTION TIPS 2 & 3 TO BE ON*/
DO2(2)=1 /*TRIGGERING SUCTION TIPS 2 & 3 TO BE ON*/
MOVE P, P99, Z=0 /*MOVE TO BAD PART BIN*/
DO2(0)=0 /*TRIGGERING SUCTION PORT 1 TO BE OFF*/
DO2(2)=0 /*TRIGGERING SUCTION TIPS 2 & 3 TO BE OFF*/
95
DO2(1)=0 /*TRIGGERING SUCTION PORTS TO BE OFF SO THAT PART IS DROPPED*/
ELSE /* CHECKING IF THE PART IS GOOD*/
GOTO *MAIN
END IF
GOTO *MAIN
96
Vita
AdityaChilukuri was born on October 25, 1987 in Hyderabad, India. The younger son of
Subrahmanyam Chilukuri and Lalitha Chilukuri, he graduated from Koneru Lakshmaiah University,
Vijayawada, India with a Bachelor of Technology degree in Mechanical Engineering in spring of 2008.
He entered University of Texas at El Paso in fall 2008 to continue his study with Master of Science in
Industrial Engineering. While at the university, he worked as a research assistant at Intelligent Systems
Engineering Laboratory and as teaching assistant in Statistics, Design of Experiments courses in
Industrial Engineering Department. He also was a member of Indian Society of Technical Education. He
has technical certifications in Six Sigma from Institute of Industrial Engineers, Lean Manufacturing
Certificate from Texas Manufacturing Assistance Center.
Permanent address: 1121 Los Angeles Dr
El Paso, Texas 79902
This thesis was typed by Aditya Chilukuri.