1. Software Engineering Practices for Large-Scale PHP Projects
PHP Con 2002, /
6/5/2018
Software Engineering Practices for Large-Scale PHP Projects
J. Scott Johnson
The FuzzyGroup
Thursday October 24, 2002, PHP-CON
3:30 – 5:00
Copyright (C) 2002, The FuzzyGroup, Inc.,

2. Table of Contents Who Am I? The Most Important Thing
What’s Different About Large Scale Projects
General Large Scale Software Engineering Practices
What’s Different About PHP?
Practical PHP for Large Projects
Stupid Pet Tricks
Credits / Conclusion / Q&A / Reading List

3. PHP Con 2002, www.php-con.com / www.phpj.com
6/5/2018
About J. Scott Johnson
15 year high tech veteran
Entrepreneur / Engineer / Product Manager
Former VP of Engineering for a dot com
Shipped > 30 versions of software from DOS PCs to Multi Server, Multi Platform Enterprise Apps (> 3.2 m lines of C++)
Currently running a profitable internet consulting firm
Author of O’Reilly’s Essential Blogging
Copyright (C) 2002, The FuzzyGroup, Inc.,

4. Not Much Code Here What NOT How
Warning !!!
Not Much Code Here
What NOT How

5. The Most Important Thing to Realize
In a Large Scale Project

6. Code Never, Ever Dies Code lives forever
Bad and Insecure code lives even longer
Doing things poorly will bite you in the ass (hard)

7. What’s Different About “Large Scale” Projects

8. 2 Aspects of What’s Different
Conceptual
Practical

9. Conceptual Large scale projects are designed
n > 1 where n might be:
Servers
Developers
Testers
Designers AND Programmers
Lots of Code
Abstraction Layers

10. Conceptual - 2 Engineering Professionalism Version control
Design notes
Code reviews
Knowledge capture
Bug tracking
Test plans

11. Practical Event Logging Throughout The Code
Error Handling Throughout The Code
Dynamic Error Alerting (to administrators)
Redundancy
0 Data Loss
Translation
Human Understandable Error Messages
Separate the HTML from your code
Separate the text from your code
Documentation

12. Software Engineering is Software Engineering !!!

13. The Language May Differ
The techniques are the same

14. General Large Scale Software Engineering Practices

15. THE MOST IMPORTANT PART OF ANY PROJECT
Design, Design, Design
THE MOST IMPORTANT PART OF ANY PROJECT
Don’t whip out code
Design takes time and needs “settling”
Write it down and THINK about it
Flexibility – Everything morphs, allow for it
Know the inputs and outputs

16. Design - 2 Build versus NIH Get other engineers to review it
Step by step, Bit by bit, Decompose the Problem
Look at both big and small pictures (but code small and build up)
Nothing is ever designed well enough
You’ll never be happy

17. Design -- Know What You Want
What problem are you trying to solve?
Who’s your target customer?
What’s their platform / browser?
What really matters to them?
Talk to real live users and potential users
Talk to internal folks at your company who really do the work – they know what actually happens
Don’t get carried away by sexy things – focus on the problem being solved

18. Write Specifications Short spec is better than no spec
Update your specifications
Tricks #1
Make specifications part of your client deliverables
Why?
Means you’ll actually write them #2
Don’t allow code check-ins w/o specs attached

19. Coding Standards You have to have them Don’t write them from scratch
Find an Open Source project you like and steal and modify
The Drupal Project is where I stole mine.
“Time bind” the amount of time spent writing them – this is a 2 to 4 hour activity!!!
Lead developer creates this and limits discussion
Keep them up to date as the project changes

20. Elements of a Coding Standard
Indentation
Brace Placement
Naming
Function Usage
Class Usage
Comments
Use of Include Files
General Guidelines
SQL

21. Coding Stds - Predictable Name Spaces
Anything is FINE but consistency is key
check_security();
checksecurity();
CheckSecurity();
Check_Security();
Predictable name spaces
Saves time
Makes code easier to read
Makes less bugs

22. Make sweeping changes early
Example
NEVER move your db access library at version 0.9
Don’t change your PHP version
This takes discipline and foresight

23. Understand How to Assign Version #s
Every release has to have a version #
Art not science
No one’s ever happy
Go to .0 when you majorly break things
Even if there are few to no new features
People don’t read squat but a .0 stops blind installs
Don’t be so complex it takes pages of documentation to understand it

24. Staffing Varies based on project Need a clear technical leader
Leader has to be able to make decisions
Staff roles
Lead engineer
Engineers
Designers
QA / Testers
Documentation

25. Database Review your schema regularly AND near the end of the project
Are fields big enough
Are fields too big? Int(11) is pretty damn large
Is your schema easily extensible for the future? IT WILL CHANGE
Will schema work cross database
Review the test data in your tables looking for truncated test data

26. Maintain a Change Log 1 line at least.
12/17/ /home/fuzzygroup.net/db.php – moved to Pear database library
Why?
It’s professional
People (you) forget
Web apps are strewn across lots of files
May get this from your version control system
Track your HTML changes too !!!

27. Version Control Absolutely Required Always useful
Indispensable if # developers > 1
Really have to do it to learn it

28. Testing Test Early Test Often Test as you go
Set a ratio of testers to coders
Have to tell the testers what changed
Not just Line 12 to 36, foo.php, added typecasting
“Try and add data and look for incorrect math”
Testers need to write test plans
Testers need to re-test bugs in the db before ship
There needs to be a bugs database

29. Testing – 2 Automate the test procedures Have a test data set
Stress test the application programmatically
Building testing tools is part of the engineering process – not an after thought
Requires discipline

30. Don’t Shoot these 4 “Guys”
Every project ships because of a small # of people
Example – Linux Kernel
Alan Cox
Linus
Keep those guys happy and you ship
Shoot them and you die

31. Bugs Bugs are GOOD They show progress No Bugs = No Progress
Don’t be embarrassed
Can’t hide them anymore

32. Use A Blog as a Documentation Tool
Blog = Weblog = Daily Updated web page
Great tool to track project changes
Make notes
Capture knowledge
Easy to update
Web form based

33. Knowledge Capture Places where knowledge exists Email!!!
Instant Messages
Document Files
Bookmarks
Capture as much of this as possible before the project ends

34. Leading / Bleeding Edge Technologies
Newer the tech, greater the bugs
Newer the tech, more likely to delay ship

35. Think About the Maintainers
They might not have your brilliance !
Coding “elegance” isn’t always good

36. Shipping and Dates Dude… We need a date.
Dates drive everything. Everything.
Expect dates to slip.
Internal calendar and external calendar.
Set Realistic Expectations.
Missing dates is demoralizing.
Don’t waste time in the beginning. Every day, hour counts.

37. PHP Con 2002, www.php-con.com / www.phpj.com
6/5/2018
Will It Ship On Time?
Even She Doesn’t Know
Copyright (C) 2002, The FuzzyGroup, Inc.,

38. Just Say No…
If (add_feature) {
cut_other_feature OR shipping_date++ }

39. Comments Not going there We’ll all just disagree anyway but …
Comments are good
Header in every source file w/ name and purpose

40. What’s Different About PHP?

41. Very Loosely Typed
You can typecast but not say that a variable is a particular type
Definite source of errors

42. Widely Used By Non-Pros
Like Basic was in the 70s / 80s
Leads to large scale apps of interesting quality
Assume a novice will try and extend your code
Comment heavily
Document well
Write FAQs
Or answer lots and lots of dumb s

43. More Complete Source Code to Use
Other languages you re-use functions
In PHP you re-use whole applications

44. Portablility Extremely Hard to Write Portable Code
Increasingly tied to a particular server configuration like
magic quotes
register_globals
Document your server settings
Windows versus linux
Filesystem issues

45. Practical PHP for Large Projects

46. Move to PHP 4.2 or later DO NOT TURN ON register_globals
Expect all your existing code to break
Expect your ISP / host to all of a sudden turn these on when you least expect it
And all code will break

47. Magic Quotes = Spawn of the Devil
Turn off Magic Quotes
Magic Quotes = Spawn of the Devil
Expect problems when you do this for code NOT your own
Expect your ISP / host to all of a sudden turn these on when you least expect it

48. Turn Off Short Tags I.e. <? Becomes <?php
This prevents XML errors so <? isn’t getting parsed into PHP

49. Centralize All Passwords
But Change Them Regularly
Do not use a .inc file – use a .php file
Why? .inc files often aren’t server parsed and are a security breach if their location is known
Lots of locations are known with common open source code

50. Using Code NOT Your Own Always read the readme Always read the FAQ
Always assume it’s not secure
Always search Google for “appname security”
See #3.

51. Objects and Classes Good but decreases performance Not magic
Useful for Design Patterns
Don’t go overboard
Objects are a technique Not a religion

52. The Church of Objects Dear Father, Who is an object,
Hallowed be thy classes
Thy constructors come …

53. Methodologies
Look at XP
Again – not a religion

54. When Your Application is “Done”
Backup
httpd.conf
php.ini
your SQL structure
AND check them into version control
Move it to a different hosting service to see if it works

55. Translation, Translation, Translation
Web site or Web Application?
Translation really matters for a Web application
phpMyAdmin
Web based
Portal toolkit
Options
Gnu gettext
My database driven approach
Home rolled approach
Dude !!! It’s Not Just English !!!

56. Text, Text, Text Get all text out of the php
Treat text as resources from a rsrc file / db
Needs to be reviewed, updated and changed by a doc author
Or you end up making the changes yourself
You’ll need to build a resource editor

57. Templates, Templates, Templates
Large projects always end up with coders and designers
Designers will break your code
Template Libraries
FastTemplate
Older
Not “supported”
Still Good
Smarty
Very sophisticated
Very powerful
Larger learning curve
Roll your own

58. Event Loops Versus N Scripts
Most code I write starts with
include XXX
switch($action) {
case login:
login();
break;
case logout:
logout();
default:
display(); }
login.php Script:
switch statement
function login() { }
function logout () {}
function display () {}
HTML
login.php?action=login
login.php?action=logout
login.php
(displays the login page)

59. Intermingling PHP and HTML
I’d say
“No. Please God No.”
Hard for anyone other than the author to maintain
Leads to a “mish mash of different pages”
See previous slide

60. Performance and Scaling
Comes at the end not the beginning or middle
Don’t over optimize or cache too early
BENCHMARK the bottlenecks
Don’t assume you know what’s happening
80 / 20 rule
Consider making a C module for core processing

61. Performance and Scaling - 2
Know the limits of your servers
Sessions are heavy
Move the static stuff to another server
Database
Move to a separate DB server
Is a DB connection limit holding you back ?
Move to persistent connections
Look at your SQL

62. Performance and Scaling - 3
PHP Con 2002, /
6/5/2018
Performance and Scaling - 3
Represent data as best for the COMPUTER not yourself
If your datasets can get large then TEST with large datasets
Copyright (C) 2002, The FuzzyGroup, Inc.,

63. Other Use wddx for serializing variables
Very useful for data blobs when no SQL is needed for specific data members
WDDX data blobs can go in the database

64. Stupid Pet Tricks

65. Scott’s Stupid Pet Tricks
Keep File Naming Conventions “Editor Optimized”
Lots of white space between functions
A sign of bad design is constantly forgetting things. IMHO.
Embed a UNIQUE identifier where you need to return. “CAT” is not unique. “JSJ” is (3 consonants don’t occur in English
Prefix your DB tables with something unique I.e. phpbb_

66. Credits

67. “With a Little Help from My Friends”
Keith Devens – Database
Ray Hunter – Overall and Design Patterns
Pete Jenney – Overall and Design Patterns
Andy Edmonds – WDDX
Mike Cannon-Brookes – Testing and Performance
Kasia Trapszo – Design, Overall
Demitrious Kelley – Design and Scaling
Kjartan Mannes – Design, Overall, Practical Tips /
Russell Beattie – Scaling

68. Conclusion

69. Conclusion & Closing Thoughts
PHP is a tool
Software engineering is a process
Only learn software engineering by doing it

70. Q & A

71. Recommended Reading Don’t Make Me Think by Steve Krug
The Mythical Man Month by Fred Brooks
Rapid Development: Taming Wild Software Schedules by Steve McConnell
Debugging the Development Process : Practical Strategies for Staying Focused, Hitting Ship Dates, and Building Solid Teams by Steve Maguire
The Inmates Are Running the Asylum : Why High Tech Products Drive Us Crazy and How To Restore The Sanity by Alan Cooper