Transfer Learning

There are 3 main research problems in transfer learning [3]:

• What to transfer
• How to transfer
• When to transfer

In tradition approaches, there are 3 main approaches (Li [1]), for domain adaptation in NLP:

• Feature space transformation
• Prior-based adaptation
• Instance selection and weighting

In modern approaches, there are 4 popular approaches (Ruder [2]):

• Pre-trained CNN features
• Learn domain-invariant representations
• Making representations more similar
• Confusing domains

[1] Li, Qi. “Literature Survey: Domain Adaptation Algorithms for Natural Language Processing,” n.d., 54. pdf [2] Transfer learning on ruder.io [3] Pan, Sinno Jialin, and Qiang Yang. “A Survey on Transfer Learning.” IEEE Transactions on Knowledge and Data Engineering 22, no. 10 (October 2010): 1345–59. https://doi.org/10.1109/TKDE.2009.191.