1. Learning through Advice-Seeking via Transfer Phillip Odom, Raksha Kumaraswamy, Kristian Kersting and Sriraam Natarajan

2. Motivation Active Advice-Seeking

3. Motivation Advice-based Learning Emphasis placed on expert: Understand the domain Be familiar with the training data Identify inconsistencies in data vs domain

4. Emphasis placed on expert: Understand the domain Be familiar with the training data Identify inconsistencies in data vs domain Goal: Utilize related domain Transfer knowledge to assist expert Motivation Advice-based Learning Related Domain

5. Outline Background Learning through Advice Seeking via Transfer (LAST) algorithm Experimental Results Conclusion

6. Language-bias Transfer Learning GIVEN: source domain knowledge small target domain data target domain ontology TO DO: generate knowledge in target domain small amount of movie data Kumaraswamy et al. ICDM 2015

7. Mode-match Trees

8. companyEconomicSector(comp, sect) ^ companyEconomocSector(comp2, sect) ^ agentHoldsSharesInCompany(per,comp2) => companyHasShareHolder(comp, per) companyEconomicSector(comp, sect) ^ personInEconomicSector(per, sect) => companyHasShareHolder(comp, per) Source: SportsQuery: teamPlaysSport Rules member(team, league) ^ member(team2, league) ^ plays(sport, team2)  teamPlaysSport(team, sport) onTeam(athlete, team) ^ athletePlaysSport(athlete, sport) => teamPlaysSport(team, sport) Source: FinanceQuery: companyHasShareHolder Rules Kumaraswamy et al. ICDM 2015

9. Relational Advice Odom et al. AAAI 2015

10. Knowledge-based Probabilistic Logic Learning Data Predictions - Gradients = Induce Iterate Final Model = ++ + + … ψmψm Advice Effect + (1-α) α Combination of Advice and Data Odom et al. AAAI 2015

11. Outline Background Learning through Advice Seeking via Transfer (LAST) algorithm Experimental Results Conclusion

12. Advice-Seeking for Transfer Combining transfer learning with advice-based learning Transfer Learning – Generates reasonable advice Advice-based Learning – Expert refines and improves the knowledge Advice-based Learning Related Domain Transfer Learning

13. LAST Algorithm Step 1: Transfer Learning Transfer knowledge from source to target to bootstrap advice Based on the relational type matching (mode matching) algorithm Step 2: SelectBestN Select top rules from transfer learning as advice Step 3: Improve Potentially allow the expert to improve the rules Step 4: Query Query the expert for the appropriate label preferences Step 5: Learn Apply KBPLMs to learn a robust model

14. LAST Algorithm Step 1: Transfer Learning Source1: paper(p1, per1), paper(p1, per2), student(per2) => advisor(per1, per2) Target1: movie(m1, per1), movie(m1, per2), actor(per1) => workedunder(per1,per2) … Step 2: SelectBestN movie(m1, per1), movie(m1, per2), actor(per1) => workedunder(per1,per2) Step 4: Query Example to which advice appliesPreferred LabelAvoided Label movie(m1, per1), movie(m1, per2), actor(per1)workedunder¬workedunder ¬(movie(m1, per1), movie(m1, per2), actor(per1))¬workedunderworkedunder

15. LAST Algorithm Step 1: Transfer Learning Source1: paper(p1, per1), paper(p1, per2), student(per2) => advisor(per1, per2) Target1: movie(m1, per1), movie(m1, per2), actor(per1) => workedunder(per1,per2) … Step 2: SelectBestN movie(m1, per1), movie(m1, per2), actor(per1) => workedunder(per1,per2) Step 4: Query Example to which advice appliesPreferred LabelAvoided Label movie(m1, per1), movie(m1, per2), actor(per1)workedunder¬workedunder ¬(movie(m1, per1), movie(m1, per2), actor(per1))¬workedunderworkedunder Can the expert improve this advice?

16. LAST Algorithm Advice without improvement Advice improvement1 Advice Improvement2 Example to which advice appliesPreferred LabelAvoided Label movie(m1, per1), movie(m1, per2), actor(per1)workedunder¬workedunder ¬(movie(m1, per1), movie(m1, per2), actor(per1))¬workedunderworkedunder Example to which advice appliesPreferred LabelAvoided Label movie(m1, per1), movie(m1, per2), actor(per1), director(per2) workedunder¬workedunder ¬(movie(m1, per1), movie(m1, per2), actor(per1), director(per2)) ¬workedunderworkedunder Example to which advice appliesPreferred LabelAvoided Label movie(m1, per1), movie(m1, per2), actor(per1), actor(per2) ¬workedunderworkedunder

17. Outline Background Learning through Advice Seeking via Transfer (LAST) algorithm Experimental Results Conclusion

18. Experimental Domains DomainPrediction IMDBWhich actors work under which directors CoraWhich conferences have the same venue UWWhich professor advises a student WEBKBThe department for which a person works NELL – Finance Which financial sector a company belongs NELL – SportsWhich sport a team plays

19. Learned Rules WEBKB -> UW Transfer Example: WEBKB: linkTo(link, site1, site2), student(site1), department(site2) -> worksFor(site1, site2) “Students link to the department in which they are enrolled” UW: taughtBy(class1, per1, sem1),yearInSchool(per2, year1) -> advisedBy(per2, per1) “Students are advised by professors”

20. Learned Rules NELL Transfer Example: NELL-Finance: acquired(company1, company2), sector(sect1, company1) -> sector(sect1, company2) “Companies make acquisitions of companies in their same financial sector” NELL-Sports: teamplaysteam(team1, team2), plays(team1, sport1) -> plays(team2, sport1) “Teams that play each other play the same sport”

21. Experimental Results

22. Conclusion Advice-Seeking via Transfer effectively unites transfer learning with human-in-the-loop learning Transfer learning can be leveraged to generate useful advice Expert’s role can be reduced, increasing effective use of their time

23. Future Work Create an interactive interface to guide the transfer process Transfer and aggregate knowledge from multiple sources Combine input from multiple experts Weight knowledge from different sources by estimated quality