1. In the paper Figure 7, we claimed: “the best value for R depends on the amount of training data available.” Here are the results for Gun-Point Dataset and another dataset, Two_ Pat, which we randomly pick half size instances repeatedly. The observation is that with fewer objects in the dataset, the accuracy decreases and peaks at larger window size. Gun PointTwo_Pat

2. Name# class# features# instancesEvaluationData type JF2220,0002,000/18,000real Letter261620,0005,000/15,000mixed Pen Digits101610,9927,494/3,498real Forest Cover Type754581,01211,340/569,672real Iris3415010-fold CVreal Ionosphere23435110-fold CVreal Voting Records21643510-fold CVBoolean Australian Credit21469010-fold CV6 numerial/8 categorical German Credit2241,00010-fold CVreal Leaf6150442200/242time series Two_Pat41285,0001,000/4,000time series Face161312,2311,113/1,118time series In the paper Table 4, we list the datasets used in the paper, here we present additional datasets and show all the experiments that could not fit in the paper due the limit of space.

3. 0200400600800100012001400160018002000 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Train Random Test SimpleRank Train SimpleRank Test 0200400600800100012001400160018002000 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test JF, 2 classed, 20,000 instances, 2,000/18,000

4. 5000050010001500200025003000350040004500 20 30 40 50 60 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Train Random Test SimpleRank Train SimpleRank Test Letter, 26 classes, 20,000 instances, 5,000/15,000 5000050010001500200025003000350040004500 20 30 40 50 60 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test

5. 01000200030004000500060007000 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Train Random Test SimpleRank Train SimpleRank Test 01000200030004000500060007000 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test Pen digits, 10 classed, 10,992 instances, 7,494/3,498

6. 0200040006000800010000 30 40 50 60 70 80 90 Number of instances seen before interruption, S accuracy(%) Random Train Random Test SimpleRank Train SimpleRank Test 0200040006000800010000 30 40 50 60 70 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test Forest Cover Type, 7 classes, 581,012 instances, 11,340/569,672

7. 0100200300400500600 70 80 90 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test Australian Credit, 2 classes, 690 instances, 10-fold Cross Validation Number of instances seen before interruption, S 0100200300400500600 40 50 60 70 80 90 100 data instances accuracy(%) RandomTrain RandomTest SimpleRankTrain SimpleRankTest DROP1 DROP2 DROP3

8. 050100150200250300 50 60 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Train Random Test SimpleRank Train SimpleRank Test 050100150200250300 50 60 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test 050100150200250300 50 60 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test BestDrop Test 050100150200250300 40 50 60 70 80 90 100 data instances accuracy(%) RandomTrain RandomTest SimpleRankTrain SimpleRankTest DROP1 DROP2 DROP3 Ionosphere, 2 classes, 351 instances, 10-fold Cross Validation

9. Iris, 3 classes, 150 instances, 10-fold Cross Validation 050100150200250300 50 60 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Train Random Test SimpleRank Train SimpleRank Test 050100150200250300 50 60 70 80 90 100 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test 020406080100120 40 50 60 70 80 90 100 data instances accuracy(%) RandomTrain RandomTest SimpleRankTrain SimpleRankTest DROP1 DROP2 DROP3

10. 050100150200250300350 90 100 Number of instances seen before interruption, S accuracy(%) Random Train Random Test SimpleRank Train SimpleRank Test 050100150200250300350 90 100 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test Voting records 050100150200250300350 90 100 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test BestDrop Test 050100150200250300350 40 50 60 70 80 90 100 data instances accuracy(%) RandomTrain RandomTest SimpleRankTrain SimpleRankTest DROP1 DROP2 DROP3

11. 0100200300400500600700800900 60 70 Number of instances seen before interruption, S accuracy(%) Random Train Random Test SimpleRank Train SimpleRank Test 0100200300400500600700800900 60 70 Number of instances seen before interruption, S accuracy(%) Random Test SimpleRank Test German Credit, 2 classes, 1,000 instances, 10-fold Cross Validation 0100200300400500600700800900 40 50 60 70 80 90 100 data instances accuracy(%) RandomTrain RandomTest SimpleRankTrain SimpleRankTest DROP1 DROP2 DROP3

12. Two_Pat, 4 classes, 5,000 instances, 1,000/4,000 split

13. Leaf Dataset, 6 classes, 442 instances

14. 02004006008001000 20 30 40 50 60 70 80 90 100 Number of instances seen before interruption, accuracy(%) Face Dataset Random, Euclidean distance Random, Fixed R = 4 SimpleRank, Fixed R = 4 SimpleRank, AdaptiveR Random, Euclidean distance Random, Fixed R = 3 SimpleRank, Fixed R = 3 SimpleRank, AdaptiveR 3% 4% Face dataset, 16 classes, 2,231 instances, 1,113/1,118 split