Presentation is loading. Please wait.

Presentation is loading. Please wait.

Why Python and other dynamic languages are a new way of thinking about programming and why its interesting to me and possibly you if.

Similar presentations


Presentation on theme: "Why Python and other dynamic languages are a new way of thinking about programming and why its interesting to me and possibly you if."— Presentation transcript:

1 Why Python and other dynamic languages are a new way of thinking about programming and why its interesting to me and possibly you if

2 youre at all like me or if youre different than me but just happen to share that same specific interest as me or if you like progress and new ideas and stuff. by Chris Gahan

3 First of all… Python and other dynamic languages (LIKE RUBY) are super-hot right now. Python and other dynamic languages (LIKE RUBY) are super-hot right now. How hot? How hot? IMAGINE A JULY DAY IN HELL AT HIGH NOON AND THE WEATHER ALSO HAPPENS TO BE ESPECIALLY HUMID THAT DAY FOR SOME REASON IMAGINE A JULY DAY IN HELL AT HIGH NOON AND THE WEATHER ALSO HAPPENS TO BE ESPECIALLY HUMID THAT DAY FOR SOME REASON Thats hot. Thats hot.

4 You want some proof? Ill give ya proof!!

5 Cool Python Sh…stuff. FuseFS FuseFS PyGame PyGame PyOpenGL PyOpenGL PyMedia PyMedia PaiMei PaiMei Pyro (Python Robotics) Pyro (Python Robotics) Matplotlib, SciPy Matplotlib, SciPy Fann Fann PIL (Python Imaging Library) PIL (Python Imaging Library) PyGTK, Tkinter, wxPython, wax PyGTK, Tkinter, wxPython, wax Googles cluster Googles cluster sCrAmBlEd?HaCkZ! sCrAmBlEd?HaCkZ! Orange (component-based data mining software) Orange (component-based data mining software) Scapy (packet masticator) Scapy (packet masticator) Twisted (everything, a kitchen sink, and a side of fries) Twisted (everything, a kitchen sink, and a side of fries) CherryPy, SQLObject (web applications) CherryPy, SQLObject (web applications) BeautifulSoup BeautifulSoup ElementTree ElementTree Psyco (optimizer) Psyco (optimizer) PyPy PyPy Pyrex Pyrex Maya, blender Maya, blender Plone Plone Amarok Amarok Subversion, Trac Subversion, Trac And many, many, many more…

6 Why is it being used for so many different applications? Its AWESOME GLUE! Mmm.. delicious glue…

7 What makes a good glue language? High-level-ousity High-level-ousity You spend more time thinking about what you want to do than how you want to do it You spend more time thinking about what you want to do than how you want to do it Python does the least to get in the way of you solving the problem Python does the least to get in the way of you solving the problem Excellent for text mastication Excellent for text mastication Text manipulation was well thought out in the language design Text manipulation was well thought out in the language design Strings have handy methods like.startswith() and.endswith() and.split() Strings have handy methods like.startswith() and.endswith() and.split() They can be sliced They can be sliced List comprehensions List comprehensions doing transformations on large chunks of data is efficient doing transformations on large chunks of data is efficient Dictionaries Dictionaries fast lookups of strings fast lookups of strings Powerful regular expression library Powerful regular expression library No more hundreds-of-lines-of-IF-statements! No more hundreds-of-lines-of-IF-statements!

8 What makes a good glue language? The built-in libraries let you do almost everything you want… easily! The built-in libraries let you do almost everything you want… easily! Any computer that has Python installed contains almost all the modules you need to do whatever you want! Any computer that has Python installed contains almost all the modules you need to do whatever you want! No sysadmin dependency, no installing 9832749823742 packages No sysadmin dependency, no installing 9832749823742 packages Can call (and wrap) code written in other languages Can call (and wrap) code written in other languages Can combine disparate systems Can combine disparate systems Connectors for every db format Connectors for every db format Full access to Win32 API (COM, registry, all that…) Full access to Win32 API (COM, registry, all that…) Excellent network protocol support Excellent network protocol support Easy to write C extensions Easy to write C extensions Maturity Maturity Has been used in tons of industries so that there are a slew of libraries floating around that let you do whatever you want Has been used in tons of industries so that there are a slew of libraries floating around that let you do whatever you want Stability (New versions dont break all your code) Stability (New versions dont break all your code) Speed! (Pythons built-in functions are heavily optimized in C) Speed! (Pythons built-in functions are heavily optimized in C)

9 What makes a good glue language? Dynamicicity Dynamicicity You can load code modules at runtime You can load code modules at runtime You can execute strings that came from somewhere else as Python programs You can execute strings that came from somewhere else as Python programs You can do generative programming You can do generative programming You can introspect your code You can introspect your code This is a very cool feature. I think it needs its own slide. This is a very cool feature. I think it needs its own slide.

10 INTROSPECTION

11 Youd better have a good reason for wasting a slide there, buddy! Introspection is an incredibly powerful feature when combined with a dynamic language Introspection is an incredibly powerful feature when combined with a dynamic language Introspection allows a program to look at itself Introspection allows a program to look at itself If the program can look at itself, and the program can create itself dynamically, then you can write a program that programs itself! If the program can look at itself, and the program can create itself dynamically, then you can write a program that programs itself!

12 Why the hell would you want to make a program that writes itself? Because youre lazy! Because youre lazy!

13 Isnt laziness bad? Short-term laziness is bad Short-term laziness is bad Long-term laziness is good Long-term laziness is good

14 Why? Good-lazy programmers are lazy in that they invent tools to do their work for them Good-lazy programmers are lazy in that they invent tools to do their work for them Achieving laziness means improving your technology, or tools Achieving laziness means improving your technology, or tools Improving your tools improves your productivity Improving your tools improves your productivity The reasonable man adapts himself to the world; the unreasonable man persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man. -- George Bernard Shaw

15 Variation in Productivity In a low-tech society you don't see much variation in productivity. If you have a tribe of nomads collecting sticks for a fire, how much more productive is the best stick gatherer going to be than the worst? A factor of two? Whereas when you hand people a complex tool like a computer, the variation in what they can do with it is enormous. […] (contd)

16 Variation in Productivity […] Productivity varies in any field, but there are few in which it varies so much. The variation between programmers is so great that it becomes a difference in kind. I don't think this is something intrinsic to programming, though. In every field, technology magnifies differences in productivity. I think what's happening in programming is just that we have a lot of technological leverage. But in every field the lever is getting longer, so the variation we see is something that more and more fields will see as time goes on. And the success of companies, and countries, will depend increasingly on how they deal with it. -- Paul Graham

17 Technology is what made this country great! If youve read the book Guns, Germs, and Steel, you know that the only reason white people conquered the world is because of all the awesome technologies they stole If youve read the book Guns, Germs, and Steel, you know that the only reason white people conquered the world is because of all the awesome technologies they stole Gunpowder, writing, math, metallurgy, the centralized state, horses, livestock, spaghetti… Gunpowder, writing, math, metallurgy, the centralized state, horses, livestock, spaghetti… basically, everything you need in the game Civilization to get to the nuclear age. basically, everything you need in the game Civilization to get to the nuclear age. The only thing they didnt steal was germs The only thing they didnt steal was germs although, we did acquire them from livestock… so I guess white people really are useless! although, we did acquire them from livestock… so I guess white people really are useless!

18 BUT ANYWAYS…

19 When do you need a program to program itself? A good example is when you do the same thing over and over and over and over. A good example is when you do the same thing over and over and over and over.

20 DRYness A good rule of thumb is the DRY principle: A good rule of thumb is the DRY principle: Dont Repeat Yourself

21 Okay, yes, you can fix the repetition by just putting all the things in a list and iterating over it like this: Okay, yes, you can fix the repetition by just putting all the things in a list and iterating over it like this: But there an easier way than manually specifying every element… But there an easier way than manually specifying every element… If you can see a pattern, then you can program a computer to see it as well. If you can see a pattern, then you can program a computer to see it as well. Couldnt you fix that without introspection?

22 You could use locals() locals() returns all of the variables in the local scope as a dict locals() returns all of the variables in the local scope as a dict Instead of specifying each thing, you could automatically grab all things like this: Instead of specifying each thing, you could automatically grab all things like this:

23 You could also improve debug messages Debugging by putting print statements everywhere is tedious, and if you want to change how theyre all displayed, you have to change every single one! Debugging by putting print statements everywhere is tedious, and if you want to change how theyre all displayed, you have to change every single one! And what if you only want debug sometimes? And what if you only want debug sometimes? Why not make a debug function that figures out the name of the function it was used in? Why not make a debug function that figures out the name of the function it was used in?

24 You could also improve debug messages Heres what it looks like in the code: Heres what it looks like in the code: (rest of the code omitted)

25 You could also improve debug messages $ python snooch.py -v http://slashdot.org/ =========================================== snooch-o-matic 9000 pro elite pro v9.1 xp ------------------------------------------- Getting all links from: http://slashdot.org/ [snooch_urls_from_page] page url = http://slashdot.org/ [snooch_urls_from_page] url_is_url(found_url): False [snooch_urls_from_page] Found: 'http://slashdot.org/http://www.ostg.com' [snooch_urls_from_page] URL IS FILE! [same_directory] base = ('http', 'slashdot.org', '/', '', ''), other = ('http', 'slashdot.org', '/http://www.ostg.com', '', '') [snooch_urls_from_page] url_is_url(found_url): False [snooch_urls_from_page] Found: 'http://slashdot.org/http://sf.net' [snooch_urls_from_page] URL IS FILE! [same_directory] base = ('http', 'slashdot.org', '/', '', ''), other = ('http', 'slashdot.org', '/http://sf.net', '', '') etc.. And here it is in action

26 How else can you DRY up your code?

27 Mr. Bot What if you wanted to write an IRC bot? What if you wanted to write an IRC bot? IRC bots have a bunch of actions that you execute like this: !do_something IRC bots have a bunch of actions that you execute like this: !do_something When youre writing a program, you could have a whole crapload of if statements that check for the command, or a series of switch statements. When youre writing a program, you could have a whole crapload of if statements that check for the command, or a series of switch statements. But what if the program could figure out what commands the bot supports by looking at the names of the methods in the bots class? But what if the program could figure out what commands the bot supports by looking at the names of the methods in the bots class?

28 Mr. Bot Heres a chunk of code from Mr. Bots logic core (the class where all !actions are coded): Heres a chunk of code from Mr. Bots logic core (the class where all !actions are coded):

29 Unittest The unittest module also discovers methods by name and runs them as tests with pretty output: The unittest module also discovers methods by name and runs them as tests with pretty output:

30 Creating new Classes automatically Also known as metaclasses Also known as metaclasses A metaclass is a class that can change what happens when you subclass another class A metaclass is a class that can change what happens when you subclass another class A good example is SQLObject A good example is SQLObject SQLObject creates a new classes that let you access your database as if its a Python object SQLObject creates a new classes that let you access your database as if its a Python object These classes are created on the fly, at runtime These classes are created on the fly, at runtime This saves a lot of typing – you just supply the data needed and no more. This saves a lot of typing – you just supply the data needed and no more.

31 Now that the computer is writing most of the code, what do us humans do?! Now that youve become a meta- programming ninja, you dont have to code as much. Youve gotta spend all your time THINKING! Now that youve become a meta- programming ninja, you dont have to code as much. Youve gotta spend all your time THINKING!

32 You bastard! We had it so good! Whyd you have to go and ruin it? I know, I know, it seems unfair, but its a cruel world out there. I know, I know, it seems unfair, but its a cruel world out there. If we dont become more productive, someone else is going to, like those sneaky JAPANESE! Then were screwed. If we dont become more productive, someone else is going to, like those sneaky JAPANESE! Then were screwed.

33 So, how do we improve Thinking? What about Lean Thinking! What about Lean Thinking! See how the letters are leaning? Thats called a visual aid.

34 Whats Lean Thinking? The basic concept is this: The basic concept is this: Do less activities that are a waste Do less activities that are a waste Do more activities that are valuable Do more activities that are valuable Examples of things that are a waste: Examples of things that are a waste: Paperwork Paperwork A bureaucratic organization produces mostly paperwork and little value. Paperwork generally gets in the way of actually doing things. A bureaucratic organization produces mostly paperwork and little value. Paperwork generally gets in the way of actually doing things. Surplus inventory Surplus inventory Produce products Just-in-Time instead of making tons and tons of them, transporting them thousands of miles to a warehouse, waiting 3 months, then transporting them all back. Produce products Just-in-Time instead of making tons and tons of them, transporting them thousands of miles to a warehouse, waiting 3 months, then transporting them all back.

35 Whats Lean Thinking Another thing thats not valuable is overly detailed program specifications Another thing thats not valuable is overly detailed program specifications The only true representation of a program is its source code. The only true representation of a program is its source code. Eventually, all specifications must be transcribed as code, and thats the thing that gets run. Eventually, all specifications must be transcribed as code, and thats the thing that gets run. Its easier to use the code as your One True Reference instead of constantly converting between code and specifications. Its easier to use the code as your One True Reference instead of constantly converting between code and specifications.

36 Wait, keeping it all in your head? Well, okay, not ALL of it… Well, okay, not ALL of it… Dont document everything – just the things that the code is bad at describing. Dont document everything – just the things that the code is bad at describing. Use cases Use cases Keep only what you need on paper. Dont overdo it. Keep only what you need on paper. Dont overdo it. Planning is good, but a lot of the time, actually implementing something changes your plans. Planning is good, but a lot of the time, actually implementing something changes your plans. Only plan things that youre absolutely certain will not change. Only plan things that youre absolutely certain will not change. For everything else, learn as you go! For everything else, learn as you go!

37 Cool! I like lean thinking! Lean Thinking is just the beginning… Some prolific software design process dudes got together in 2001 and created the Agile Manifesto. Lean Thinking is just the beginning… Some prolific software design process dudes got together in 2001 and created the Agile Manifesto. The Agile Manifesto contains the current ideas on the best way to produce software. The Agile Manifesto contains the current ideas on the best way to produce software. Its goals are Its goals are maximal quality maximal quality maximal programmer satisfaction maximal programmer satisfaction minimal time minimal time equal cost equal cost

38 Agile Development Agile tries to keep focus on creating a process that allows each individual in each team to find and use their unique strengths Agile tries to keep focus on creating a process that allows each individual in each team to find and use their unique strengths Flexibility Flexibility Dynamism Dynamism Learning Learning Trying new ideas Trying new ideas Adaptability and learning are essential Adaptability and learning are essential Continuous improvement of: Continuous improvement of: the people the people the process the process the source code the source code

39 Agile Development Agile has a lot to do with optimizing the efficiency of people Agile has a lot to do with optimizing the efficiency of people Humans are the bottleneck, not computers! Humans are the bottleneck, not computers! Maximizing human efficiency is tricky Maximizing human efficiency is tricky Humans are pattern recognizers, not state machines Humans are pattern recognizers, not state machines This means that people kinda suck at reading and imagining code This means that people kinda suck at reading and imagining code We can understand small pieces at a time, but not an entire system in all its gory detail. We can understand small pieces at a time, but not an entire system in all its gory detail.

40 Agile Development Techniques to help improve the efficiency of programmers Techniques to help improve the efficiency of programmers Writing tests Writing tests Tests let you use your module before youve written it Tests let you use your module before youve written it Keeps you secure in your code so you dont have to worry about it Keeps you secure in your code so you dont have to worry about it Short release cycles Short release cycles Get your program running and interfacing with real-world data as soon as possible Get your program running and interfacing with real-world data as soon as possible Helps keep you focussed on whats important Helps keep you focussed on whats important Keep in constant communication with team members and customer Keep in constant communication with team members and customer If the every member of the team doesnt know the state of the program, then theyre going to be afraid to change parts that theyre not responsible for, and theyll also make mistakes when integrating their code with the code of others If the every member of the team doesnt know the state of the program, then theyre going to be afraid to change parts that theyre not responsible for, and theyll also make mistakes when integrating their code with the code of others If the customer isnt kept in the loop, then the project could start going in the wrong direction. The customer is the end-user and ultimate judge of the finished product – their feedback is the most important. If the customer isnt kept in the loop, then the project could start going in the wrong direction. The customer is the end-user and ultimate judge of the finished product – their feedback is the most important. Pair programming Pair programming Two brains each doing different specialized tasks Two brains each doing different specialized tasks brain #1: tactical [writing code] brain #1: tactical [writing code] brain #2: strategic and verification [thinking about architecture, verifying what brain #1 is doing] brain #2: strategic and verification [thinking about architecture, verifying what brain #1 is doing] Keeps the programmers completely synchronized – two people will always be 100% up to speed on a certain part of the program Keeps the programmers completely synchronized – two people will always be 100% up to speed on a certain part of the program

41 PyPy has taken Agile to the extreme: Sprints! A PyPy Sprint is a two or three-day focused development session in which developers pair off together to focus on building a particular subsystem A PyPy Sprint is a two or three-day focused development session in which developers pair off together to focus on building a particular subsystem It ends up being incredibly productive, and is really the best way to produce high-quality code It ends up being incredibly productive, and is really the best way to produce high-quality code The alternative is to work slowly, unfocussedly, individually. The alternative is to work slowly, unfocussedly, individually. Humans dont work well like that! Our memories fade over time. Humans dont work well like that! Our memories fade over time.

42 Why does nCircle exist? Because software has tons of flaws! Because software has tons of flaws! Thank the maker for bad development practices and tools. Thank the maker for bad development practices and tools.

43 Why does software have tons of flaws? Because people are stupid! Because people are stupid!

44 Dont be glib… Well, okay, programming beyond a certain level of complexity is too mentally taxing for humans. Well, okay, programming beyond a certain level of complexity is too mentally taxing for humans.

45 Fundamental Problems with Software With the advent of faster computers, and with society pushing inexorably towards the refinement and automation of all things, software complexity has shot through the roof With the advent of faster computers, and with society pushing inexorably towards the refinement and automation of all things, software complexity has shot through the roof Programmers are digging themselves into holes all over the place Programmers are digging themselves into holes all over the place Making their lives more complicated than it needs to be Making their lives more complicated than it needs to be

46 Basically… We know how we want software to behave, but we just dont have the mental hardware (or the software tools) to allow us to continuously juggle a massive nest of interconnected logical constraints and remember every single one. We know how we want software to behave, but we just dont have the mental hardware (or the software tools) to allow us to continuously juggle a massive nest of interconnected logical constraints and remember every single one. We get tired! We get hungry! We get distracted by that videos of the monkey peeing into his mouth! We get tired! We get hungry! We get distracted by that videos of the monkey peeing into his mouth!

47 The Future of Programming Jonathan Edwards says: I discovered the creative exhilaration of programming at a formative age. Yet all along I have also felt stifled by primitive, ill-suited languages and tools. We are still in the Stone Age of programming. My quest is to better understand the creative act of programming, and to help liberate it. Reflection upon my own practice of programming has led me to three general observations: Jonathan Edwards says: I discovered the creative exhilaration of programming at a formative age. Yet all along I have also felt stifled by primitive, ill-suited languages and tools. We are still in the Stone Age of programming. My quest is to better understand the creative act of programming, and to help liberate it. Reflection upon my own practice of programming has led me to three general observations: Programming is mentally overwhelming, even for the smartest of us. The more I program, the harder I realize it is. Programming is mentally overwhelming, even for the smartest of us. The more I program, the harder I realize it is. Much of this mental effort is laborious maintenance of the many mental models we utilize while programming. We are constantly translating between different representations in our head, mentally compiling them into code, and reverse- engineering them back out of the code. This is wasted overhead. Much of this mental effort is laborious maintenance of the many mental models we utilize while programming. We are constantly translating between different representations in our head, mentally compiling them into code, and reverse- engineering them back out of the code. This is wasted overhead. We have no agreement on what the problems of programming are, much less how to fix them. Our field has little accumulated wisdom, because we do not invest enough in the critical self-examination needed to obtain it: we have a reflective blind-spot. We have no agreement on what the problems of programming are, much less how to fix them. Our field has little accumulated wisdom, because we do not invest enough in the critical self-examination needed to obtain it: we have a reflective blind-spot.(contd)

48 The Future of Programming From these observations I infer three corresponding propositions: From these observations I infer three corresponding propositions: Usability should be the central concern in the design of programming languages and tools. We need to apply everything we know about human perception and cognition. It is a matter of Cognitive Ergonomics. Usability should be the central concern in the design of programming languages and tools. We need to apply everything we know about human perception and cognition. It is a matter of Cognitive Ergonomics. Notation matters. We need to stop wasting our effort juggling unsuitable notations, and instead invent representations that align with the mental models we naturally use. Notation matters. We need to stop wasting our effort juggling unsuitable notations, and instead invent representations that align with the mental models we naturally use. The benefits of improved programming techniques can not be easily promulgated, since there is no agreement on what the problems are in the first place. The reward systems in business and academics further discourage non-incremental change. The benefits of improved programming techniques can not be easily promulgated, since there is no agreement on what the problems are in the first place. The reward systems in business and academics further discourage non-incremental change.

49 Ruby + Domain Languages Managing complex projects is about abstractions Managing complex projects is about abstractions Your programs API is like its own domain-language Your programs API is like its own domain-language Good abstractions make writing programs easier because we can mentally manipulate them better Good abstractions make writing programs easier because we can mentally manipulate them better A change of perspective is worth 80 IQ points. – Alan Kay A change of perspective is worth 80 IQ points. – Alan Kay Intuitive paradigms are the most efficient way to allow a human to interface with a computer Intuitive paradigms are the most efficient way to allow a human to interface with a computer The best programming paradigms are very intuitive: The best programming paradigms are very intuitive: Lists Lists Stacks Stacks Dictionaries Dictionaries Memory space Memory space Objects with Actions Objects with Actions Threads Threads The implications of calling a function should be intuitive The implications of calling a function should be intuitive Ruby allows you to make your API easier to understand by making it easy to create domain languages Ruby allows you to make your API easier to understand by making it easy to create domain languages Ruby almost allows you to extend the language itself Ruby almost allows you to extend the language itself eg: ActiveRecord, Choice eg: ActiveRecord, Choice A standard API will have a lot of boilerplate. In Ruby, its easy to make all that boilerplate disappear. A standard API will have a lot of boilerplate. In Ruby, its easy to make all that boilerplate disappear.

50 Intentional Programming This is probably going to be the first step on a road towards an evolving system that allows humans to represent programs on their own terms. This is probably going to be the first step on a road towards an evolving system that allows humans to represent programs on their own terms. Think of the Star Trek computer Think of the Star Trek computer


Download ppt "Why Python and other dynamic languages are a new way of thinking about programming and why its interesting to me and possibly you if."

Similar presentations


Ads by Google