Forest Inventory, LiDAR, and Patents Jim Flewelling Presented at Silvilaser 2009, College Station, TX, Oct. 14-16, 2009 Seattle Biometrics and Analysis, LLC flewelling seattlebiometrics.com Minor corrections made November 2, 2009

IF YOU WANT TO OBTAIN A PATENT  PAY ATTENTION ! IF YOU WANT TO PREVENT A PATENT  PAY ATTENTION ! IF YOU WANT TO BE SUED FOR INFRINGEMENT  TAKE YOUR MORNING SIESTA !

THANK THIS MAN FOR NOT PATENTING “FOREST INVENTORY BY LASER” ROSS NELSON Terje Gobakken

Dedicated to an Inventor Walter Bitterlich (1908-2008)

80th, in 1988, 3 Relascopes Photo by Kim Iles

Bitterlich – several inventions 2. Angle Gauge 1. Angle-count Sampling G = k × (Tree Count) not patented 4. Model for critical height sampling and growth. 3. Relascope, w Benno Hesske Theories from Kitamura, and Iles.

OUTLINE Patent Basics Disclaimer Inventory Methods Patents Ways to Influence Patents Patent Infringement Resources

Patent Basics Patents exclude others from using the patented product or method. Patents have one or more claims. Each claim is a “mini patent”. Each claim has one or more elements.

Patent Basics Patent Applications are Submitted by all the true inventors. Novel. Unobvious. Useful. Of a patentable nature. Priority date: usually date the patent application is made.

Patent Basics Prior art are publications or public disclosures before the priority date.

Patent Basics / Obligations Duties apply to inventors and others. Continuing duty of candor: to disclose all known material related to patentability. (Slideshow by Moatz, 2007) : http://www.ipo.org/AM/Template.cfm?Section=Calendar&Template=/CM/Co ntentDisplay.cfm&ContentID=15913

Disclaimer Nothing shown here is intended as a criticism of the inventors. Potential problems of the patents are raised. Speaker has no legal background. For your issues, get a good IP attorney.

OUTLINE Patent Basics Disclaimer Inventory Methods Patents Ways to Influence Patents Patent Infringement Resources

Inventory Methods Patents Inventors Priority Date Key word in title Application # or US Patent Assignee Hyyppä 1999.10.28 Stand Attributes (ALS) Patent 6792684 Diware Oy Rousselle et al 2002.05.03 Feature identification (photo) 7212670 1st: Falcon, now ImageTree Corp. Kelle et al 2006.06.19 Feature Identification Prov. Appl 60/814715 Probabilistic Sampling Application 11/728099 Bortolot & McTague 2006.08.16 Forest Inventory (tree clusters, ALS) 11/505189 Abandoned. Hyyppä and Yu 2003.03.31 Growth of Trees 10/803966 Geodeettinen Laitos Savolainen et al 2004.03.15 (photo + lidar) 10/598322 Blom Karita Oy, Geodeettinen L.. Jupp et al 2001.02.09 Lidar System (ALS + Terrestrial) 7187452 CSIRO (Australia) Welty et al 2007.06.22 Vegetation Attributes 7474964 Weyerhaeuser Company

Hyyppä, J. Priority Date 1999.10.08 Title Method for Determination of Stand Attributes and a Computer Program to Perform the Method Literature Hyyppä, J, Hyyppä, H and A. Samberg (1999, prior). Hyyppä, J, and M. Inkinen (1999, subsequent) Hyyppä et al. (2001)

Hyyppä, J. Priority Date 1999.10.08 First of 20 Claims Method for stand estimation. Steps 1 and 2: obtain ALS data. Step 3: Make a crown surface model or anything similar. Step 4: Make any sort of stand estimates by building from single trees or groups of trees. “Method for determination of stand attributes with an instrument above the stand, comprising: moving a laser scanner in a first direction, collecting three-dimensional information from a stand by sweeping a laser beam of the laser scanner in a second direction substantially perpendicular to the first direction and by using such a number of sample hits that individual trees or groups of are identified, producing a three-dimensional presentation of a stand height from the collected three-dimensional information, and determining stand attributes, being characteristics of individual trees or groups of trees and characteristics for larger stand areas, from the three-dimensional presentation”

Hyyppä, J. Priority Date 1999.10.28 Potential Problem – Not Novel and Unobvious Nilsson (1996): “ If airborne laser data could be given both height and planimetric coordinates with high accuracy, single trees or groups of trees could be identified and detailed terrain models could be generated. this would be of great importance in many different situations, ...” Borgefors, G., T. Brandtberg, and F. Walter, (1999) delineated tree crowns from ALS data and noted that heights were known for the pixels within each crown and could be used to estimate tree heights. Brandtberg’s dissertation posted 1999.10.28 (too late!) St-Onge published in November, 1999, but 1997 grant proposal states: “The height of individual trees will be obtained from a local maximum technique also based on model-based segmentation in order to locate tree tops.” Pre-October 1999 grant proposals by Hyyppä, Borgefors, Andersen, Young Evans and Parker , Popescu, Koch and Friedlander ??????

Hyyppä, J. Priority Date 1999.10.08 Patent Family Austria Australia Canada Germany Europe Spain Finland Norway U.S. World From patentlens.net Initiative of Cambia (AU), free.

“Method of Feature Analysis” Literature: McCrystal (2003) Rousselle, A., Leppanen, V., McCrystal, D., Kelle, O. and R. Pliszka Priority Date 2002.05.03 “Method of Feature Analysis” Literature: McCrystal (2003) Crown delineation based on photo pixels. Manual training procedure. Thirty-nine claims. Many steps in each. “Data model” to predict tree size based on crown size.

Rousselle, et al. Priority Date 2002. 05 Rousselle, et al Priority Date 2002.05.03 Technological Limitation – Inappropriate Models. “Data model” is a regression model fit to subjectively selected trees, using ground-measured crown sizes. Hyyppa’s “generally known formulas” to determine diameters are similar. Sampling theory is not being used. “.. Jim helped us understand that our methods of compiling this precise data leave it biased and unreliable” Adam Rousselle, personal communication. August, 2005.

Rousselle, et al. Priority Date 2002. 05 Rousselle, et al Priority Date 2002.05.03 Example of change in inventorship. Original Application filed in May, 2002. In December, 2004, the USPTO is petitioned to name Robert Pliszka as an inventor. (Granted). All must agree that he is a true inventor, and that his original omission was not due to deceptive intent on their parts. Note: a patent knowingly filed with incorrect inventorship is void ab initio.

Kelle, O. , Macom, E. , Pliszka, R. , and N Kelle, O., Macom, E., Pliszka, R., and N. Mathawan Priority Date 2006.06.19 Provisional Application Title Method of Feature Identification and Analysis Law Provisional Applications do not become patents. They merely establish the subject matter, inventorship, and priority date. Why Provisional applications are filed when an inventor is not yet ready to fully describe what it is he wants to claim. He may be concerned that someone else will invent or publish on the same method.

“Description of the method.” Kelle, O., Macom, E., Pliszka, R., and N. Mathawan Priority Date 2006.06.19 Provisional Application “Description of the method.” “Lidar and multispectral imagery are being used separately or together to identify and characterize individual tree crowns. Operationally it is feasible to collect data for entire forests, and to construct digital maps with stand boundaries and with polygons representing tree crown. Techniques to create crown polygons, assign species and impute tree characteristics are the subject of many ongoing research efforts in Scandinavia, the U.S. and elsewhere. A starting point for sampling can be the map with the crown polygons attributed with species and possibly LIDAR height; this is used as a sampling frame for a statistically valid sampling inventory. The sample design ….”

Title Date What Literature Kelle, O., Macom, E., Pliszka, R., and N. Mathawan Priority Date 2006.06.19 Regular Application Title Remote Sensing and Probabalistic Sampling Based Forest Inventory Method. Date Filed on March 23, 2007 What This application claims the priority of the earlier provisional application. The subject matter is little changed. Now claims are stated. Literature Flewelling (2006) Flewelling (2008) at Silvilaser 2008 Flewelling (2009)

Kelle, O. , Macom, E. , Pliszka, R. , and N Kelle, O., Macom, E., Pliszka, R., and N. Mathawan Priority Date 2006.06.19 First Claim, Regular Appl. A remote sensing and probabilistic sampling based forest inventory method comprising: a. processing remote sensing data indicative of tree attribute information for said forest, said remote sensing data comprising at least one of LiDAR data and digital images; b. defining a sampling frame within said remote sensing data; c. determining a field plot corresponding to said sampling frame and collecting field plot data therefrom, said field plot data comprising actual tree attribute information; d. generating a correlated model by combining said field plot data with said remote sensing data corresponding to said sample frame; and e. applying said correlated model to all said remote sensing data to produce a probabilistic forest inventory. Compare with “Description of Method” in provisional application.

Kelle, et al. Priority Date 2006.06.19 Potential Problem – Not novel. June 19, 2006 presentation by Flewelling covers similar material. (Not prior art). Abstract distributed on June 18 may be prior art. Named inventors may publish up to one year before a U.S. patent application. This may preclude getting a European patent.

Details of an application. USPTO.gov Enter the eBusiness tab / Public PAIR Enter application number or publication number or patent number. Bibliographic data includes current status. Transaction history shows what has happened. Documents easy to download. (except NPL)

eBusiness. Status & View Documents (Public Pair) CLICK HERE 

Thirty-five claims applied for. Most are rejected; Six may be OK. Kelle, O., Macom, E., Pliszka, R., and N. Mathawan STATUS, HOT OFF THE INTERNET Thirty-five claims applied for. Most are rejected; Six may be OK. Attorney cancels 32 claims. Examiner approves patent for issuance with three claims. Another named inventor is now shown.

Inventory Methods Patents – Next Five Inventors Priority Date Key word in title Application # or US Patent Assignee Bortolot & McTague 2006.08.16 Forest Inventory (tree clusters, ALS) Application 11/505189 Abandoned. Hyyppä and Yu 2003.03.31 Growth of Trees (ALS) 10/803966 Geodeettinen Laitos Savolainen et al 2004.03.15 Stand Attributes (photo + lidar) 10/598322 Blom Kartta Oy, Geodeettinen Jupp et al 2001.02.09 Lidar System (ALS + Terrestrial) Patent 7187452 CSIRO (Australia) Welty et al 2007.06.22 Vegetation Attributes 7474964 Weyerhaeuser Company There are many reasons that patents are applied for. Sometimes this is useful even if the plan is to abandon them.

OUTLINE Patent Basics Disclaimer Inventory Methods Patents Ways to Influence Patents Patent Infringement Resources

Ways to Influence Patents Defensive Publishing (*) Publish early. SCRIBD.com – instant publications. File your own patent applications. “Foresee” a lot. Silvilaser proceedings. SCIRUS.com can include. Include in a regular publication series. Petition USPTO to include in field of search. * Adams et al. (2002)

Ways to Influence Patents Call attention to prior art. Submit to patent office. Within 60 days of published appl. (U.S.) After patent is issued. Submit to attorneys and inventors* While patent is pending anywhere. * See Cortina (2007)

OUTLINE Patent Basics Disclaimer Inventory Methods Patents Ways to Influence Patents Patent Infringement Resources

Patent Infringement Research Use of Patented Methods * Countries Permitted ? EU, Norway, Japan, New Zealand, Korea, Mexico. Probably Australia, Canada Uncertain United States Probably Not * See Dent et al (2006). Do not rely on this table.

Patent Infringement Your Defense Willfull Infringement Not practicing the method. Claims are invalid. You have a license Entire patent is invalid due to inequitable conduct. Willfull Infringement Treble damages in the U.S. Get an attorney’s written opinion first. In the U.K., patent office may issue opinions.

OUTLINE Patent Basics Disclaimer Inventory Methods Patents Ways to Influence Patents Patent Infringement Resources

Resources WEB SITE Why Uspto.gov Most info on pending patents Patentstorm.com Easy, copies of patents PatentlyO.com Legal forum Gnu.org Public licensing Gauss.ffii.org European patents Groups.fsf.org Software issues

Resources / seattlebiometrics.com My site currently empty. U.S. patent publications by next week. May start with ftp. Would discussion forums be useful?

Inventorship and Teaching “Courtesy and appreciation of the work of others are two of the reliable hallmarks of those in the top ranks of our profession”. Kim Iles (2003, p. 261). These qualities are difficult to discern in many of today’s patent applications. Walter Bitterlich and Kim Iles

References not included in the Proceedings Borgefors, G., T. Brandtberg, and F. Walter, 1999. Forest parameter extraction from airborne sensors. In; Proceedings of the ISPRS Conference Automatic Extraction of GIS Objects from Global Imagery (eds: Ebner, H., Eckstein, W., Heipke, C. and H. Mayer), ISPRS Conference, Sept. 8-10, 1999, Munich, Germany International Archives of Photogrammetry and Remote Sensing Vol. 32, Part 3-2W5, pp 151-158. (camera ready paper submitted May, 1999; proceedings distributed at the conference?) Brandtberg, T. 1999. Automated individual tree-based analysis of high spatial resolution remotely sensed data. Center for Image Analysis, Uppsala, Sweden. Thesis. [November, 1999, “nailed” on October 28.] Cortina, A. J. 2007. The duty to disclose: withhold at your patent’s peril. LocalTechwire (May 30, 2007). http://localtechwire.com/business/local_tech_wire/opinion/story/1454653/ Iles, K. 2003. A Sampler of Inventory Topics. Kim Iles & Associates, Nanaimo, BC, Canada, 869 p. (ISBN 0-9732198-0-7. www.island.net/~kiles )

References not included in the Proceedings HYYPPA , J., SCHARDT, M., HAGGRE´N, H., KOCH, B., LOHR, U., SCHERRER, H.U., PAANANEN, R., LUUKKONEN, H., ZIEGLER, M., HYYPPA¨ , H., PYYSALO, U., Friedlander, H., UUTTERA, J., WAGNER, S., INKINEN, M., WIMMER, A.,KUKKO, A., AHOKAS, E. and KARJALAINEN, M., 2001, HIGH-SCAN: The first European-wide attempt to derive single-tree information from laserscanner data. Photogrammetric Journal of Finland, 17, pp. 58–68. Koch B., and Friedlaender H., 1999, Erste Erfahrungen mit dem Einsatz von Laserscannerdaten zur Erfassung von vertikalen und horizontalen Strukturen im Wald, In Albertz J. (Hrsg.): Publikationen der Deutschen Gesellschaft für Photogrammetrie und Fernerkundung e.V., Band 8, pp. 335-343, Tagungsband der 19. Wissenschaftlich-Technischen Jahrestagung der DGPF, Essen, 13-15 October 1999. [I have not yet found a copy of this ] St-Onge, B.A. 1999. Estimating individual tree heights of the boreal forest using airborne laser altimetry and digital videography. La Jolla Conference, 9-11 November, 1999 (Commission III). p 174-184. Young, B., Evans, D.L., Parker, R.C. 2000. Methods for comparison of lidar and field measurements of loblolly pine. In proceedings: Second International Conference on Geospatial Information in Agriculture and Forestry, Lake Buena Vista, Florida, 10-12 January 2000: I-193-199. (CD distributed at the conference)