ARTIFICIALNEURAL NETWORKS, 2

Proceedings of the 1992 International Conferenceon Artificial Neural Networks (ICANN-92)

Brighton, United Kingdom, 4-7 September, 1992

Edited by

IgorALEKSANDERDepartment of Electrical and Electronic Engineering

Imperial College of Science Technology and MedicineLondon, United Kingdom

and

John TAYLORDepartment of Mathematics

King's College LondonLondon, United Kingdom

Volume 1

UNIVERSITATS3IBUOTHEKHANDOVER

TECHNISCHEINFORMATIOHSBIBLIOTHEK

1992

NORTH-HOLLANDAMSTERDAM • LONDON • NEW YORK • TOKYO

Vll

Table of ContentsVOLUME 1

Plenaries

Artificial Neural Networks: Models, Pardigms or Methods ? 3T. Kohonen

From Single Neuron to Cognition 11J. G. Taylor

Capturing Consciousness in Neural Systems 17/. Aleksander

Associative-memory models of the Cerebellum 23P. Kanerva

Analogue Neural VLSI: Issues, Trends and Pulses 35A. F. Murray

Why are Neural Networks so Wide ? 45S.Judd

Combining Two Methods of Recognizing Hand-printed Digits 53G.E.Hinton, C.KJ.Williams, M.D.Revow

Neurocontrol: Where it is Going and Why it is crucial 61PJ.Werbos

Conference Papers

Learning Theory 1Organizer: HJ. Kappen

Learning Rules, Stochastic Processes, and Local Minima 71{Invited paper)B. Kappen, T.Heskes

The Final Prediction Error Criterion for MLP Models 79A. Varfis

Imposing A Generalisation Strategy on Artificial Neural Networks 83GD. Tattersall, G.E. Lee

A Self-organized Locally Tuned Network for Coarse Coding 87G. Deco, J. Ebmeyer

Vlll

Reaching the Generalization Maximum of Back-Propagation Networks 91W. Schoner

On Systematically Diluted Hopfield Networks 95K.-R. Muller

Global Performance of Learning Rules 101T. M. Heskes, E.T.PSlijpen

Generalization of Back-Propagation Based on Tellegens Theorem 105G. Tyma

Computational Limitations on Training Sigmoid Neural Networks 109K.-U. Hoffgen

Network Topology, Training Set Size and Generalization Ability in MLP's Project 113P. Burrascano

Temporal Knowledge in Locations of Activations in a Self-Organizing Map 117J. Kangas

Unsupervised Symmetry Detection: A Network which Learns from Single Examples 121W. Konen, C.von der Malsburg

A Learning Threshold Algorithm for Binary Linear Codes 127A. Esposito, S. Rampone, R. Tagliaferri

The Use of an Adaptive Distance Measure in Generalizing Pattern Learning 131G. Scheler

On Bounded-depth Threshold Circuits for Pattern Functions 135A. Albrecht

The Bearable Lightness of Being: Reducing the Number of Weights in 139Backpropagation NetworksS. Santini

A Reinforcement Learning Algorithm for Networks of Units with Two Stochastic Levels 143D. Kontoravdis, A. Likas, A. Stafylopatis

Learning Theory 2Organizer: L.B. Almeida

Adaptive Decorrelation (Invited Paper) 149L.B. Almeida, F.M. Silva

Training Window Neurons Through the Solution of Linear Equations 157M. Muselli

Sequential Learning of Two-layer Networks with the Incomplete Upstart Algorithm 161M. Muselli

An Ordering Theorem that Allows for Ordering Changes 165EA. Ferrdn

A Complexity Metric and Benchmark Set for Learning Continuous Functions 171R.W. Green, CM. DeAngelis

Symmetries of General Feedforward Neural Networks and Equivalent Classification 175TasksP. DeWilde, A.M.C.-L. Ho

Unsupervised Hebbian Learning and the Shape of the Neuron Activation Function 179J.L. Shapiro, A. Priigel-Bennet

Ockham's Nets: Self-Adaptive Minimal Neural Networks 183GD. Kendall, TJ.Hall

The Generalisation Ability of Dilute Attractor Neural Networks 187C. Campbell

Attractor Structure of Constrained Neural Networks at Finite Temperature 191C. Campbell

How to Remember the Future: Temporal Asymmetry in Mechanisms of Synaptic Change 195O. Holland, M.Snaith

PC A in Fully Parallel Neural Networks 199E. Oja, H. Ogawa, J. Wangviwattana

Study on Minimal Net Size, Convergence Behaviour and Generalization Ability of 203Heterogeneous Backpropagation Network/. Kamruzzaman, Y. Kumagai, H. Hikita

A New Geometric Approach to the Design of Neural Associative Memory 207/. Dehaene, J. Vandewalle

The Howl-effect in Dynamic-network Learning 211C. Thornton

A Theory of Over-Learning 215H. Ogawa, K. Yamasaki

Deterministic Nets and Learning 3Organizer: E.R. Caianiello

Do all Neurons Know Algebra? 221{Invited Paper)EJi. Caianiello

On the Capacity of n-h-1 Networks with Sigmoidal Functions 229M. Yamasaki, A.Sakurai

Improving Generalization Performance by Nonconvergent Model Selection Methods 233W. Finnoff, F. Hergert, H.G. Zimmermann

On the Capacity of n-h-s Networks 237A. Sakurai, M. Yamasaki

A Signal Sharing Scheme for Unifying Computation and Learning Phases of Recurrent 241Networks

/. Kumazawa

Assigning Two Modes of Computation to Cells of a Formal Neural Network for 245Unsupervised Learning of Sequences

B. Gas, R. Natowicz

A Connectionist Model Constrained by an Optical Implementation 249F. Alexandre, F. Guyot

Cascade LLM Networks 253E. Littmann, H. Riuer

Reinforcement Learning and Subtasks 259T. Wengerek, H. Ritter

Designing Neural Networks Using Structured Genetic Algorithm 263D. Dasgupta, DK. McGregor

M-RCE: A Self Configuring ANN with Rule-extraction Capabilities 269N. Tschichold-Gurman, M. Ghazvini, D. Diez

Learning A-posteriori Probabilities with Multi-layer Perceptron Classifiers 273M.JJ. Holt

The Connections of Large Perceptrons 277W. Wiegerinck, T. Coolen

Stable States, Transitions and Convergance in Kohonen Self-organising Maps 281DA. Critchley

Structural Adaption of Boolean Higher Order Neurons: Classification with 285Parsmonious Topologies for Superior GeneralizationG. Fahner, N. Goerke, R. Eckmiller

A Learning by Reinforcement Model from the Neurobiological Outlook 289FJ.Vico, F.Sandoval

When a Bird in a Bush is Worth Two in the Hand 293O. Holland, M. Snaith

Information Retrieval in Sparse Associative Memories 297M. Ceccarelli, A. Petrosino

Neuromathematics 1Organizer: S-I. Amari

Learning Curves, Generalization Errors and Information Criteria 305{Invited Paper)S-I. Amari

XI

Gibbs Distribution Theory of Adaptive N-Tuple Networks 313SPLuttrell

Autogenerative Nodal Memory Model (ANM) - A Mathematical Analysis of Autogenerated 317Internal Content

F.A. Monaco

Characteristic Properties of Stable Recurrent Higher Order Neural Networks 325A. Bischoff, B. Schumann, J. Maruhn, J. Reinhardt

Layer Based Neural Network Formalization 329E. Fiesler, HJ. Caulfield

Phase Space Dynamics of Neural Nets 333U. Ramacher, M. Wesseling

A Neural Network Model for the Generation of Epilectic Foci 341A. Adamopoulos, P. A. Anninos

Looking at Hopfield-type Associative Memory Through Eigenvalues Analysis and SVD 345S. Skoneczny

Non-Euclidean Cellular Automata and Neural Nets 349K. Melkas, K. Kaski

The Effect of Correlated Inputs on Discrete Kohonen Networks 353K. Ishida, Y. Matsumoto, N. Okino

On the Role of Time-varying Recurrent Synapses on the Properties of Neural Maps 359A.CJi.DaSilvaFihlo

Stability of Asymmetric Networks and Dale's Law 363H.G. Barrow

Eigenvalues and Eigenvectors for Compartmentally Modeled Unbranched Neurons 367A.J. Klaassen

Attractor Learning of Recurrent Neural Networks 371K. Gouhara, H. Takase, Y. Uchikawa, K. Iwata

Association Dynamics of Cross-Coupled Hopfield Nets with Many-to-Many Mapping 375InternetworksS. Ozawa, K. Tsutsumi, H. Matsumoto

Prediction of Chaos in Non-autonomous Systems by a Neural Network 379/. Grabec

xu

Neuromathematics 2Organizer: A. Babloyantz

From the single Neuron to the Cortical Networks - A Coherent Model of EEG 385{Invited Paper)A. Babloyantz

Boltzmann Machines with Finite Alphabet 391J. Shawe-Taylor, J. Zernovnik

A New Global Optimization Method : "Rolling-Stone Scheme" and Its Application to 395Supervised Learning of Multi-layer PerceptronsJ. Chao, W. Ratanasuwan, S. Tsujii

Cytoskeleton as Feedback Control System in Neuron 399D. Koruga, M. Andjelkovic, S. Jankovic, S. Hameroff

Equilibrium and Turbulent Solutions to Recurrent Neural Fields in R 2 403CJ3. Price, P. Wambacq, A. Oosterlinck

Approximating Functions and Predicting Time Series with MuM-Sigmoidal Basis 407FunctionsM. Benaim, L. Tomasini

Back Propagation in a Clifford Algebra 413J.K. Pearson, DL. Bisset

A Novel Architecture of High Order Associative Memory with Reduced Cross Talk 417Y. Kumagai, J. Kamruzzaman, Y. Maruyama, H. Hikita

Geometric Methods and Generalisation in Feedforward Networks 421J.Mitchell

Discovering Nonlinear Dependences in the Output of Feed-forward Networks 425A. Sperduti, A. Starita

The Bottleneck of the Bidirectional Associative Memory and Its Generalizations 429H. Englisch, M. Herrmann

Automatic Gain Control by a Basic Neural Circuit 433H.G. Barrow, JML.Budd

Combinatorial Optimization by Random Neural Network Model Application to the 437Independent Set ProblemF. Pekergin

Global Behaviour in Asymmetric Continuous Neural Networks 441J.-L. Gouze

Capacity and Spurious Memory of Associative Memory Using a Non-Monotonic 445Neuron Model5. Yoshizawa, M. Morita, S. Amari

Extraction of the Prototypes Encoded in chaotic Attractor 449A. Yu. Loskutov, V. M. Tereshko

X l l l

Sensorimotor ControlOrganizer: R. Eckmiller

Model-Based Training Method for Neural Controllers 455T. Hrycej

Rule Extraction for BNN Neural Network-based Fuzzy Control System by 459Self-Learning

DA. Linkens, J. Nie

Counterpropagation Network-Based Fuzzy Controllers: Explicit Representation and 463Self-Construction of Rule-BasesJ. Nie, DA. Linkens

Minimum-Motor-Command-Change Trajectories Predicted by Cascade Neural Network 467Using a 17- Muscle Arm ModelM. Dornay, M. Kawato. Y. Uno, R. Suzuki

Control of a Paraplegic Patient Model by Neuroprothetic Networks 471J. Beckmann, WJ. Daunicht, V. Homberg

A General Approach to Learning of Task Sequences 475F.E. Lauria, M. Sette

A Neurocontroller Based on A Learning Boolean Network 479F. Bini Verona, F. E. Lauria

A Neural Network Study on the Role of Efference Copy Feedback in the Generation 483of the Collicular Motor MapK.P. Krommenhoek, AJ. van Opstal, C.C.A.M. Gielen, J.A.M. van Gisbergen

SOBoS - A Self-Organizing Body Schema 487P. Morasso, V. Sanguineti

Voluntary Motor Control Using Biologically Plausible Neural Networks with 491Application to Advanced Biologically Inspired RobotsM.P. Norman, M.J. Denham

Vision 1Organizer: K. Fukushima

An Improved Learning Algorithm for the Neocognitron 497{Invited Paper)K. Fukushima, N. Wake

A Dynamical Multiparticle Model for Early Vision 505M. Bengtsson

Relaxation in 4D State Space - A Competitive Network Approach to Object-related 509Velocity Vector-field CorrectionH. Glunder, A. Lehmann

XIV

Performance of Hopfield ANN in Simulating Human Optical Illusions: The Muller-Lyer 513ParadigmE. Ventouras, C. Papageorgiou, N.K. Uzunoglu, A. Rabavilas, C. Stefanis

Bionocular Vergence Control and Depth Reconstruction Using a Phase Method 517W.M. Theimer, HA. Mallot

An Incremental Neural Classifier of Configurations of Active Orientation-specific Line 521DetectorsA.P. Azcarraga, B. Amy

A Precategorial Network Model for Line and Edge Detection and Texture Segregation 525M.M. Van Hulle, T. Tollenaere

Nonlinear Programming Neural Network Application to Image Filtering 529A. Dzienlinski, S. Skoneczny

A Parallel Translation of the Waltz Algorithm for Analyzing Line-drawings 533T. Tambouratzis

The Economy/Performance Issue in Harmony Theory Networks 537T. Tambouratzis

Optical Flow Estimation Algorithm for Neural Networks 541T. Watanabe, Y. Isobe

Image Recognition with a Cooperation of Symbolic Inference and Neural Networks 545N. Sakurai, T. Omori

Parallel Distributed Switching with Gating Lattices 549E.O. Postma, H. J. van den Herik, P.T.W. Hudson

Self-organization of Binocular ReceptiM. Andres, H. Mallot, G.-J. Giefing

ve Fields 553

Discovering Textures with a Metric Based on Image Structure 557G. Smith, J. Austin

Interaction of 3-D Illusory Object with Binocular Viewing 561M. Idesawa

An Efficient Neural Architecture Implementing the Boundary Contour System 565E. Ardizzone, A. Chella, G. Compagno, R .Pirrone

Robotics and ControlOrganizer: I. Aleksander

A Neural Architecture for Concurrent Generation of Configuration Space Trajectory 5715. Bhattacharya, A. Ghosal

A Neural Network Model for Trajectory Formation of Arm Movement by Using 575Forward and Inverse Dynamics ModelsY. Wada, M. Kawato

XV

Self-organizing Neural Network Apllication to Technical Process Parameters Estimation 579E. Govekar, E. Susie, P. Muzic, I. Grabec

High-precision Robot Control: The Nested Network 583A. Jansen, P.P. van der Smagt, F.CA. Groen

Building Maps for Path-planning and Navigation Using Learning Classification of 587External Sensor DataA. Kurz

A Neural Network for Collision-free Path Planning 591H. Meng, PD. Picton

Receptive Fields for CMAC. An Efficient Approach 595W.S. Mischo

Extending the Adaptive Heuristic Critic and Q-learning: From Facts to Implication 599O. Holland, M. Snaith

Neural Networks for Optimal Control of Nonlinear Stochastic Systems 603T. Parisini, R. Zoppoli

Control of Time-delay Systems Using Reinforcement Learning 607T. Langlois, S. Canu

A Neuro-controlled Autonomous Robot That Can Capture a Moving Object 611K. Tanaka, M. Shimizu, K. Tsuchiya

Node Allocation and Topographical Encoding (NATE) for Inverse Kinematics of a 615Redundant Robot Arm - Learning without a Teacher/. Hakala, H.W.Werntges, JJt. Beerhold, R. Eckmiller

Adaptive State Space Quantisation: Adding and Removing Neurons 619BJA.Krose, J.W.M.vanDam

Weightless SystemsOrganizer: I. Aleksander

An Automata-theoretic Assessment of the Cognitive debate 625{Invited Paper)I. Aleksander

Adaptive Reward-penalty for Probablistic Logic Nodes 631R.S. Neville, TJ. Stonham

A Modular Weightless Neural Network System 635R.J. Mitchell, PJi. Minchinton, J.P. Brooker, J.M. Bishop

Auto-Associative Digital Neural Network for Grey Level Data 639J.M. Bishop, R.J. Mitchell, P.R. Minchinton

XVI

Implementing Hard Self-organization Tasks Using Logical Neural Networks 643G. Tambouratzis, T.J. Stonham

A Cascadable 2048-Neuron VLSI Artificial Neural Network with On-Board Learning 647T. Hui, P. Morgan, K.Gurney, HBolouri

The Use of Encoded Outputs and Reinforcement Training in pRAM Nets 653Y. Guan, T.G. Clarkson, J.G. Taylor, D. Gorse

The Hysteretic Neuron - A Model with Energy for Learning 657P.JL. Adeodato, C.G.Fernandes

Turing Machine Simulation by Logical Neural Networks 663W.R. de Oliveira, T.B. Ludermir

SOFT - A Boolean Self-organising Feature Extractor 669A. de Carvalho, M.C. Fairhurst, DL. Bisset

A Lazy Learning Approach to the Training of GSN Neural Networks 673A. de Carvalho, M.C. Fairhurst, DL. Bisset

Optimum Selection of Class Vectors 677M.T. Brown, J. Austin

Bounding the Alphabet Size of Fixed Weight Block Codes 681R. Beale

A Rapid Coding Methodology Utilising A Distributed Associative Memory 685RBeale

Hybrid Systems For Robot PlanningOrganizer: P. Morasso

Hybrid Systems For Robot Planning 691{Invited Paper)P. Morasso, G. Vercelli, R. Zaccaria

On the Problem of Connectionist Production Systems - Models and Their Implemetation 699NX. Kasabov, St. Shishkov

Competitive Learning in Classifier Feature Maps 703NJt. Ball

Theory and Experiments in Connectionist AI: A Tightly-coupled Hybrid System 707A. Giacometti, B. Amy, A. Grumbach

Generation of Symbolic Rules in Back-Propagation Networks 711T. Denoeux

Learning Fuzzy Control Rules with a Fuzzy Neural Network 715F. dAlchi-Buc, V. Andres, J.-P. Nodal

xvu

Local Abduction and Prediction of Unit Clauses in Symmetric Hopfield Networks 721P.M.V. Lima

Semantic Transparency, Brain Monitoring and the Definition of Hybrid Systems 727J.G. Wallace, R. Silberstein, K. Bluff

Detecting Structures in Natural Language Using Neural Net and Rules 731C. Lyon, R. Frank

Knowledge Aquisition with Self-Organizing Neural Networks 735A. Ultsch

A Hybrid Method of Hidden Markov Model and Neural Network Classifier for On-line 741Handwritten Character RecognitionS.-B. Cho, J.H. Kim

A New Hybrid Approach to Robot Vision 745F. Callari, A. Chella, S. Gaglio, R. Pirrone

Approximate Reasoning with Hybrid Connectionist Logic Programming Systems 749N.K. Kasabov, S.H. Petkov

A Symbolic/Subsymbolic Interface for Variable Instantiation 753R. Stark

A Rule-based Network Architecture 757J. Hollatz, V. Tresp

Speech / Vision 2Organizer: F.Fallside

Issues in Speech Recognition Using Neural Networks (Invited Paper) 765F. Fallside

Improving Rejection in Neural Network Classifiers: An Application to 775Isolated Word RecognitionR. Gemello, F. Mana

Integration of Neural Networks and Hidden Markov Models for Continuous 779Speech RecognitionZ. Zhao

Task Decomposition Through A Modular Connectionist Architecture: 783A Talker Identification SystemY. Bennani, P. Gallinari

Classification and Reproduction of Time Sequences 787W. Simantzik, R.W. Brause

Self-organizing Map in Acoustic Analysis and On-line Visual Imaging 791of Voice and ArticulationP. Utela, J. Kangas, L. Leinonen

XVU1

Diphone Synthesis Using a Neural Network 795G.C. Cawley, PD. Noakes

An Attempt to Recognise Structural Aberrations in Chromosomes Using a Neural 799Network SystemM. Turner, J. Austin, N. Allinson, P. Thompson

Automatic Feature Selection by Self-organization 803A. Visa

Neural Networks for Low-level Image Processing 809D.T. Pham, E.J. Bayro-Corrochano

Application of Optic Flow of a Neural Network Based on the Kullback Entropy 813J.F. Boyce, SM. Protheroe, J.F. Haddon

Algebraic Methods for Learning in Stochastic Syntactic Neural Networks 817S.M. Lucas

Learning to recognize 3D-Hand Postures from Perspective Pixel Images 821A. Meyering, H. Ritter

A Neural Network Hierarchy for Data and Knowledge Controlled Selective Visual 825AttentionH.-M. Grob, E. Koerner, H.-J. Boehme, T. Pomierski

Neural Net Vector Quantizer for DCT Coding of Speech 829R. Kunchev, L. Veleva

Shape, Position and Size Invariant Visual Pattern Recognition Based on Principles 833of Neocognitron and PerceptronC. Neubauer