1. INFO 344 Web Tools And Development
CK Wang
University of Washington
Spring 2014

2. Make life simple
Read the assignment carefully! Line by line! Especially the hints/reminders section.
Start crawler button
Input is optional, life is probably easier if you disallow input, just hardcode and
We only crawl html/htm pages but these sites might still have parameters!
Keep your crawler running when submitted! PA3 is due Monday, I’ll start grading Thurs/Fri. Should have > 100k URLs

3. General Errors Forbidden access Invoke -> Page not found
Missing index.html
Invoke -> Page not found
Port issue. Make sure port matches admin.asmx
Reference errors
You need references in both projects! Web and Worker roles are separate projects. You need to add references to both.
Modify config files in both.

4. Azure Table Errors Table Insert - Conflict Error
Make sure TableEntity has a default constructor
Table Insert - Conflict Error
Means the row already exists, use intellisense to see what API you should use instead of Insert()
Table Insert – Bad Request
Try a different partitionkey, rowkey, table name
Read online on the requirements of these parameters. Such as – space ‘ ‘ is NOT ok.

5. Final Hints Draw everything on paper!
Draw Web role, Worker role
how they communicate
where data is stored (urls? Start/stop commands? Page titles? Cpu stats? Dashboard stats?)
Follow the arrows on paper and write in each web/worker what they need to do, that is your algorithm!

6. Algorithm Worker role = similar to quiz Run() Sleep 100ms
Check queue to see if I have work
If yes, do work
If no, do nothing
Loop back to #1

7. Questions?