Fuzzy aggregated topology evolution for cognitive multi-tasks

Chaturvedi, Iti, Chit, Lin Su, and Welsch, Roy E. (2021) Fuzzy aggregated topology evolution for cognitive multi-tasks. Cognitive Computation, 13. pp. 96-107.

[img] PDF (Accepted Publisher Version) - Published Version
Restricted to Repository staff only

View at Publisher Website: https://doi.org/10.1007/s12559-020-09807...
 
6


Abstract

Evolutionary optimization aims to tune the hyper-parameters during learning in a computationally fast manner. For optimization of multi-task problems evolution is done by creating a unified search space with a dimensionality that can include all the tasks. Multi-task evolution is achieved via selective imitation where two individuals with the same type of skill are encouraged to crossover. Due to the relatedness of the tasks, the resulting offspring may have a skill for a different task. In this way, we can simultaneously evolve a population where different individuals excel in different tasks. In this paper, we consider a type of evolution called Genetic Programming (GP) where the population of genes have a tree like structure and can be of different lengths and hence can naturally represent multiple tasks.

Methods : We apply the model to multi-task neuroevolution that aims to determine the optimal hyper-parameters of a neural network such as number of nodes, learning rate and number of training epochs using evolution. Here each gene is encoded with the hyper parameters for a single neural network. Previously, optimization was done by enabling or disabling individual connections between neurons during evolution. This method is extremely slow and does not generalize well to new neural architectures such as Seq2Seq. To overcome this limitation, we follow a modular approach where each sub-tree in a GP can be a sub-neural architecture that is preserved during crossover across multiple tasks. Lastly, in order to leverage on the inter-task covariance for faster evolutionary search we project the features from both tasks to common space using fuzzy membership functions.

Conclusions :The proposed model is used to determine the optimal topology of a feed-forward neural network for classification of emotions in physiological heart signals and also a Seq2seq chatbot that can converse with kindergarten children. We can outperform baselines by over $10\%$ in accuracy.

Item ID: 65015
Item Type: Article (Research - C1)
ISSN: 1866-9964
Keywords: Multi-task Optimization; Genetic Programming; Fuzzy Logic; Neuroevolution
Copyright Information: © The Author(s), under exclusive licence to Springer Science+Business Media, LLC part of Springer Nature 2021
Funders: James Cook University (JCU), Nanyang Technical University (NTU)
Projects and Grants: JCU College of Science and Engineering, NTU Data Science and Artificial Intelligence Center (DSAIR)
Date Deposited: 10 Jan 2021 23:22
FoR Codes: 46 INFORMATION AND COMPUTING SCIENCES > 4602 Artificial intelligence > 460204 Fuzzy computation @ 70%
32 BIOMEDICAL AND CLINICAL SCIENCES > 3201 Cardiovascular medicine and haematology > 320101 Cardiology (incl. cardiovascular diseases) @ 15%
52 PSYCHOLOGY > 5204 Cognitive and computational psychology > 520401 Cognition @ 15%
SEO Codes: 92 HEALTH > 9201 Clinical Health (Organs, Diseases and Abnormal Conditions) > 920103 Cardiovascular System and Diseases @ 30%
97 EXPANDING KNOWLEDGE > 970108 Expanding Knowledge in the Information and Computing Sciences @ 70%
Downloads: Total: 6
Last 12 Months: 3
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page