1. Chapter 6. Attribute Data & Tables

2. Table terminology Title Field Records Each field is specifically defined and established before any data can be entered. Field definitions control the type of data that can be stored in a field.

3. Types of tables Attribute table –Stores attributes of map features –Associated with a spatial data layer –Has special fields for spatial information Standalone table –Stores any tabular data –Not associated with spatial data –OID instead of FID

4. Database Management Systems Dedicated systems for managing tables of data Provide data management for agencies, universities, companies, etc. Designed for multi-user environments with enhanced security needs Focus on data tables with tools for queries, reporting, graphing, etc.

5. Flat file DBMS Flat file –Stores data as rows of information in files –Simple and robust –Inefficient for search and query Service calls Customers Electric usage Service personnel

6. Hierarchical DBMS Stores data in multiple tables Tables have defined parent- child relationships Pre-set hierarchy of table relationships designed for specific queries Very efficient for specific queries Range of queries limited by structure Service calls Customers Electric usage Service personnel Billing history Customer ID Employee ID Customer ID

7. Relational DBMS Stores data in multiple tables Table relationships are defined as needed Very flexible Ideal for open-ended applications when queries not known beforehand Most common type used in GIS applications Customers Electric usage Service calls Service personnel Billing history Customer ID Employee ID

8. Queries- What is a query? A query extracts certain records from a table based on a specified condition For example, –How many students have GPA > 3.5? –How many states have population > 1 million? –How many counties had greater population in 1990 than in 2000? –How many customers have last names beginning with “Mac” or “Mc”?

9. Structured Query Language, or SQL Many databases use a special query language called Structured Query Language Can write queries that work in multiple DBMS environments Queries can be saved and reused Nearly always case-sensitive

10. SQL Query Examples Some Valid Queries SELECT *FROM cities WHERE "POP1990" >= 500000 SELECT *FROM counties WHERE “BEEFCOW_92” < “BEEFCOW_87” SELECT *FROM parcels WHERE “LU-CODE” = 42 AND “VALUE” > 50000 SELECT *FROM rentals WHERE “RENT” > 700 AND “RENT” < 1500 ARC GIS hasan interface to help users build SQL expressions In most databases, SQL expressions are case-sensitive “Smith” ≠ “SMITH”

11. Multiple Criteria Queries Multiple criteria queries using AND or OR “STATE” = ‘Alabama’ OR “STATE” = ‘Texas’ [Value] > 5000 AND [Value] < 10000 –Note that the field name must be repeated for each condition

12. Searching for partial matches Sometimes you need to find one string within another rather than an exact match – Find all customer names beginning with “Mac” or “Mc” – Find all zip codes beginning with 0 Typically uses a “wildcard” character “ * ” – *Mac* or *Mc* – 0*

13. The Like Operator “NAME” LIKE ‘%(D)%’ –Finds all of the (D) Democrats % is the “IGNORE” wildcard Ignores Don or Danforth “NAME” LIKE ‘%New %’ –Would find New Hampshire and New York, but not Newcastle or Kennewick

14. Query results The result of a query is a set of selected records that meet the criteria Subsequent operations on the table will use only the selected records and ignore the others. A user might… –Export the selected records to a new table –Calculate statistics on the selected records –Calculate new values for the selected records –Produce a report based on the selected records

15. Joining tables Joined table Destination tableSource table Common field

16. Join facts Joins are temporary relationships between tables used by a relational DBMS Tables must share a common field (key) Treats the two tables as a single table Original stored data is not affected Can be removed when no longer needed

17. One-to-one joins When each record in the destination table matches exactly one record in the source table, we call it a cardinality of one-to-one. Destination table (always imagine on the left) Source table (always imagine on the right)

18. Types of Cardinality One-to-one –States to Governors –Countries to capitals One-to-many –States to cities –Districts to schools Many to one –Cities to states –Schools to districts Many-to-many –Students to classes –Stores to customers In evaluating cardinality, always put the destination first. (Destination on the left)

19. Rule of Joining Each record in the destination table must match one and only one record in the source table. One to one Destination table Source table Many to one

20. One to many Destination tableSource table ? Violates the Rule of Joining Record to join to destination is ambiguous Must use a relate instead

21. No matches? Join still occurs, but values are given to records with no match Earthquakes table has records for Puerto Rico States table does not have records for Puerto Rico

22. Multiple joins You can add multiple joins to a destination table using different keys You cannot join to a joined source table Destination Source 1 Source 2

23. Relates Similar to a join except that –The tables remain separate –Items selected in one table may be highlighted in the related table

24. Related tables Select the New England states Find reps of New England states

25. Table statistics Often a first step in data analysis Opportunity to think critically about data Returns statistics only for selected set

26. Thinking critically Why are there so many zero magnitude quakes? Why so few small earthquakes? What are the implications for data analysis? Expected distribution

27. Summarizing tables Calculate statistics for groups of features in a table Groups by unique values in the one field User chooses statistics to calculate for other fields Produces another table as output with groups and stats How many people live in each subregion? What is the total area of each subregion?

28. How to summarize Right-click Subregion field Sum Pop2010 Sum SQMI Choose statistics Name output table

29. Summarize Output Table Count field always generated automatically

30. Create map Could we now create a map of population in subregions? No, there are no features. Standalone table

31. Joining the table States layer Summarized table

32. Fields Fields have specific types available Must be defined before use Once defined, cannot be changed Naming rules –No more than 13 characters –Use only letters and numbers –Must start with a letter Database storage concepts

33. Binary data Fundamental mode of computer storage Data are stored as a series of 0’s and 1’s representing numbers in base 2 (bits) Bits are grouped by eight to form 1 byte. A megabyte (MB) is 1 million bytes. 10011101 1 bit one byte In base 2: 00000000 = 0 11111111 = 255 2 8 = 256 1111111111111111 = 65,565 2 16 = 65,566 two bytes 0 1 10 11 100 101 110 111 1000 1001 1010 1011

34. ASCII storage American Standard Code for Information Interchange (ASCII) Stores letters, characters, and symbols as single 7-bit binary codes CAT = {67,65,84} decimal = 100001110000011010100 cat = {99,97,116} decimal = 110001111000011110100 148 = {49,52,56} decimal = 011000101101000111000

35. Storing data Text data always stored in ASCII format Numeric data may be stored in ASCII or binary format Binary is generally more efficient 106 = {49,48,54} decimal = 011000101100000110110 ASCII stores three letter codes of 1 byte each = 3 bytes Binary stores 106 as a single 1-byte binary number 106 = 01101010

36. Byte storage limits A single byte can store a value from 0 to 2 8 -1 Larger numbers require more bytes – 1-byte 2 8 -1 = 255 –2-bytes 2 16 -1 = 65,535 –3-bytes 2 26- 1 = 16,777,215 –6-bytes 2 32 -1 = 4,294,967,295 Signed numbers require a bit to store positive or negative, so storage limits are smaller –2 bytes 2 15 -1 = -32,767 to +32,767 –4 bytes 2 31 -1 = -2,147,483,647 to +2,147,483,647 In base 2: 00000000 = 0 11111111 = 255 2 8 = 256

37. Integer vs float storage Binary stores whole numbers (integers) To store decimal values, the computer stores a form of scientific notation with a mantissa and an exponent – 3.2957239e04 = 32957.239 – -3.2957239e04 = -32957.239 – 3.2957239e-04 = 0.00032957239 Scientific notation 3.2957239 x 10 4

38. Float precision Large numbers start to lose precision because the number of significant digits in the mantissa is limited. – 3.2957239e12 = 3295723900000 A double-precision floating point allots more storage to the mantissa value – 3.295723956249723e12 = 3295723956249.723

39. Database storage Database fields typically are defined by –ASCII vs binary type storage –Bytes of storage allocated –Integer vs floating point Definition limits the values that can be stored –Important to match type to storage requirements –Try to minimize storage space while making sure all potential values will fit in the field Text (ASCII) field with 10 bytes “Mississipp” Binary 2-byte signed integer: -32,767 to +32,767 Single-precision floating pointx.xxxxxxxeyy

40. ArcGIS field data types

41. Time to think What kind of storage would be used to store the following kinds of data? How many bytes are needed for each field? –Last name –Number of children in household –Student grade point average –Year of birth –County population –Aerial image of county

42. Tables in ArcGIS Tables contain attribute data Many formats, one interface

43. Sources of tables Dbase files INFO files ASCII Text files (tab or comma delimited) Records from SQL database systems Excel worksheets

44. ArcMap table interface Title FieldRight-click field name to get menu Records Status bar Options menu

45. Managing multiple tables Switch between Arrange in tabs

46. Table Options menu Commands affect the currently active table Active table

47. Adjusting field width Temporary, does not affect stored file Hover over field break to get double arrow, then drag

48. Field properties tab Field alias Hide field Move field up/down Show/hide all Format

49. Shortcut to field properties Right-click field name

50. Formatting field display

51. Table appearance

52. Sorting tables Has no effect on original data

53. Select by Attributes Some valid queries [POP1990] > 1000000 “STATE_NAME” = ‘Alabama’ [POP2000] >= [POP1990] DBF tables have field names enclosed in quotes Geodatabase tables have field names enclosed in brackets

54. Show selected

55. Clear Selection In Table window From main menu

56. ArcGIS field data types

57. Adding a field In ArcMap Think about the field type before you add the field!

58. Adding multiple fields Easier in the Catalog properties –Type name in empty row –Set field type –Set field properties –Click Apply –Add next… 3 1 2 Think about the field type before you add the field!

59. Editing fields Open Editor toolbarStart editing Type edits in fieldsSave edits, stop editing Editing and calculating fields

60. Calculating fields Add a new field if necessary Consider whether you need decimal places!

61. Calculate Right-click field to calculate Usually field should be EMPTY Be careful! Calculations can’t be undone unless you are in an edit session Enter expression

62. Geodatabase geometry measures Automatically created and maintained –Usually appear at end of table Shape_Area field Shape_Length field –Units will match units of the coordinate system

63. Shapefile geometry measures Shapefiles DO NOT create or maintain area/length fields automatically! –Must be created and updated manually –Some functions and operations can change the lengths and areas of features –If you find an AREA field there is no guarantee that it is correct

64. Be careful! Before After AREA/LENGTH/PERIMETE R fields in shapefiles are NOT updated when features change.

65. User geometry calculations Right-click empty or incorrect field to update Choose type, coordinate system, and units

66. Test yourself: True or False? A field named AREA in a shapefile will always have the correct area? –False. Shapefile area fields are not automatically maintained. A field named AREA in a geodatabase will always have the correct area? –False. Automatically updated fields in a geodatabase are called Shape_Area.

67. How to join or relate tables Right-click destination in the Table of Contents

68. Importing text (ascii) tables Allowed in ArcGIS a)Tab-delimited b)Comma-delimited Not allowed! c)Fixed-column Importing tables

69. Excel files Excel worksheets with suitable layout can be opened as tables in ArcGIS Most functions that do not involve changing the file will work (sort, query) Tables cannot be changed or edited

70. Excel file requirements First row must contain field headings with legal field names as defined earlier No blank rows or formulas should be used Each column should contain only text or only numbers. It is helpful for each column to be formatted as text, numeric, etc. ArcMap cannot open files that are already open in Excel—they must be closed first

71. Acceptable Worksheet Legal field names Columns formatted as text or numeric No formulas or blank lines Decimal degrees = degrees + minutes/60 + seconds/3600

72. Workbooks and worksheets An Excel workbook file (.xls) may contain more than one worksheet. By default there are three named Sheet1, Sheet2, Sheet3. There may be one or more named worksheets. ArcMap can only open one worksheet at a time. You will open the workbook like a folder and select a single worksheet

73. Adding a worksheet 1. Add data button 2. Open workbook file like a folder 3. Add named worksheet, or the first unnamed worksheet.

74. Excel file in ArcMap You can use x-y locations in a table to produce points on a map. But first you must know the coordinate system of these locations. What is this one?

75. Displaying x-y values from a table A table with x-y values can be displayed as points. Use any coordinate system, but you must know what it is. Geographic coordinate system (GCS)

76. Display XY Data Degrees indicate a geographic coordinate system (GCS). NAD83 is the most common. Other tables might have units in UTM meters, State Plane feet, etc. You must know the coordinate system to create correctly located points. Use the Edit button to define the coordinate system.

77. Oregon climate stations Stations appear as points on the map, an “event layer” Export to shapefile or feature class to keep permanently