More

Extract/copy a subset of rasters from a mosaic dataset

Extract/copy a subset of rasters from a mosaic dataset


I have a mosaic dataset of 421 images and I want to extract just a few of them to give to a client. Rather than click through the folder selecting the files (because they are not contiguous) I want to transfer is there a way to use the list of selected footprints (using selection features in ArcGIS) to automate the copying? If there is I'm running into a total mind blank and have not found anything close to what I'm trying to do.

Possibly something I could use in Total Commander 8 or CMD prompt (i.e. xcopy/robocopy, etc) because ArcGIS would be too slow I think.

EDIT.

I also have Global Mapper 14 to use of the explorer options aren't feasible.


Personally I'd create a mosaic dataset (or VRT) and extract the area of interest from that.

VRT is supported by Global Mapper but will be quite slow, I would go with the mosaic dataset for this operation.

Both of these raster types are links only to their respective rasters so don't take very long to create as the rasters themselves aren't being copied. Mosaic datasets will create overviews and will take a bit longer - but that's probably worth it.

Extracting from ArcGis can be done in a few ways:

  • Open the mosaic dataset in ArcMap and then export the dataset using the extent of the data frame.
  • Clip the raster to a known extent box using Clip_management (don't confuse with Clip_analysis which is for features)
  • Extract by mask using a shape (spatial analyst license required)

To get a list of all the rasters in all subfolders you can useDirin DOS like this:

DIR c:whereimagesare*.tif /b /s > c:somepathTiffFileList.txt

Which instructs command to find all the files with .tif extension in this folder and all subfolders and write just their full path to a file c:somepathTiffFileList.txt - obviously substitute your own paths and extension if not tif. If your paths contain spaces they will need to be quoted likeDIR "c:path with spaces*.tif" /B/S > c:somepathTiffFileList.txtor the instruction will get confused.

Then open the text file, copy the contents and paste into load rasters for a mosaic dataset or use-input_file_list c:somepathTiffFileList.txtfor GDALBuildVRT.

If you know which rasters you want to copy you can open up the file list in Excel, delete the rows that you're not interested in, insert a column and fill withcopythen save as csv - open the csv in notepad and replace the commas with spaces then save as .bat and you've got a batch file to copy rasters… put the batch file in the location you're wanting to copy the files to and double click. You might run into problems if you have duplicate file names or spaces in your paths though.


Given a mosaic dataset, you can select the footprints of interest and then in the table of contents right-click the footprint layer > Data > Download Selected Rasters… which then allows you to choose which rasters to download.


While I realize this is a very old post. I ran into a much better solution recently. Export Mosaic Dataset Items can do exactly what you are asking. You can export by query definition or clip type (Feature Class). I hope this helps anyone stumbling on this post in the future.


Reading in a subset of rasters based on filenames in R

I'm working with rasters in R. Extracting data from them to coordinates, to be exact. However, my issue is not actually in working with the rasters so much as it is with reading in the specific rasters I want to work with at any given time.

I have 35 years worth of raster data, each raster is named in a way that reflects its date. For example: "raster.01.01.1990.tif"

All rasters are in one folder. Sometimes I need to extract data from all the rasters, in that case, it's straight forward. Set the wd, create a list of the rasters, and read those rasters into R.

No problem. I can then do what I need to do. However, sometimes I only want to work with certain years. In that case, I've been excluding certain rasters from my file list based on if their filename contains the years I don't want.

This approach works however, I can't help feel that there is probably a much more elegant solution. In particular, if I just want to work with 3 years of data I then have to manually exclude 32 years worth of data. Sure it doesn't take that long to type out. but it is inefficient.

Is there a more efficient way to exclude or include files based on file names than the method I'm using above?


5.2. Raster Catalogs

There are two raster catalog views that come packaged with PostGIS. Both views utilize information embedded in the constraints of the raster tables. As a result the catalog views are always consistent with the raster data in the tables since the constraints are enforced.

raster_columns this view catalogs all the raster table columns in your database.

raster_overviews this view catalogs all the raster table columns in your database that serve as overviews for a finer grained table. Tables of this type are generated when you use the -l switch during load.

5.2.1. Raster Columns Catalog

The raster_columns is a catalog of all raster table columns in your database that are of type raster. It is a view utilizing the constraints on the tables so the information is always consistent even if you restore one raster table from a backup of another database. The following columns exist in the raster_columns catalog.

If you created your tables not with the loader or forgot to specify the -C flag during load, you can enforce the constraints after the fact using AddRasterConstraints so that the raster_columns catalog registers the common information about your raster tiles.

r_table_catalog The database the table is in. This will always read the current database.

r_table_schema The database schema the raster table belongs to.

r_table_name raster table

r_raster_column the column in the r_table_name table that is of type raster. There is nothing in PostGIS preventing you from having multiple raster columns per table so its possible to have a raster table listed multiple times with a different raster column for each.

srid The spatial reference identifier of the raster. Should be an entry in the Section 4.3.1, “The SPATIAL_REF_SYS Table and Spatial Reference Systems”.

scale_x The scaling between geometric spatial coordinates and pixel. This is only available if all tiles in the raster column have the same scale_x and this constraint is applied. Refer to ST_ScaleX for more details.

scale_y The scaling between geometric spatial coordinates and pixel. This is only available if all tiles in the raster column have the same scale_y and the scale_y constraint is applied. Refer to ST_ScaleY for more details.

blocksize_x The width (number of pixels across) of each raster tile . Refer to ST_Width for more details.

blocksize_y The width (number of pixels down) of each raster tile . Refer to ST_Height for more details.

same_alignment A boolean that is true if all the raster tiles have the same alignment . Refer to ST_SameAlignment for more details.

regular_blocking If the raster column has the spatially unique and coverage tile constraints, the value with be TRUE. Otherwise, it will be FALSE.

num_bands The number of bands in each tile of your raster set. This is the same information as what is provided by ST_NumBands

pixel_types An array defining the pixel type for each band. You will have the same number of elements in this array as you have number of bands. The pixel_types are one of the following defined in ST_BandPixelType.

nodata_values An array of double precision numbers denoting the nodata_value for each band. You will have the same number of elements in this array as you have number of bands. These numbers define the pixel value for each band that should be ignored for most operations. This is similar information provided by ST_BandNoDataValue.

out_db An array of boolean flags indicating if the raster bands data is maintained outside the database. You will have the same number of elements in this array as you have number of bands.

extent This is the extent of all the raster rows in your raster set. If you plan to load more data that will change the extent of the set, you'll want to run the DropRasterConstraints function before load and then reapply constraints with AddRasterConstraints after load.

spatial_index A boolean that is true if raster column has a spatial index.

5.2.2. Raster Overviews

raster_overviews catalogs information about raster table columns used for overviews and additional information about them that is useful to know when utilizing overviews. Overview tables are cataloged in both raster_columns and raster_overviews because they are rasters in their own right but also serve an additional special purpose of being a lower resolution caricature of a higher resolution table. These are generated along-side the main raster table when you use the -l switch in raster loading or can be generated manually using AddOverviewConstraints.

Overview tables contain the same constraints as other raster tables as well as additional informational only constraints specific to overviews.

The information in raster_overviews does not duplicate the information in raster_columns . If you need the information about an overview table present in raster_columns you can join the raster_overviews and raster_columns together to get the full set of information you need.

Two main reasons for overviews are:

Low resolution representation of the core tables commonly used for fast mapping zoom-out.

Computations are generally faster to do on them than their higher resolution parents because there are fewer records and each pixel covers more territory. Though the computations are not as accurate as the high-res tables they support, they can be sufficient in many rule-of-thumb computations.

The raster_overviews catalog contains the following columns of information.

o_table_catalog The database the overview table is in. This will always read the current database.

o_table_schema The database schema the overview raster table belongs to.

o_table_name raster overview table name

o_raster_column the raster column in the overview table.

r_table_catalog The database the raster table that this overview services is in. This will always read the current database.

r_table_schema The database schema the raster table that this overview services belongs to.

r_table_name raster table that this overview services.

r_raster_column the raster column that this overview column services.

overview_factor - this is the pyramid level of the overview table. The higher the number the lower the resolution of the table. raster2pgsql if given a folder of images, will compute overview of each image file and load separately. Level 1 is assumed and always the original file. Level 2 is will have each tile represent 4 of the original. So for example if you have a folder of 5000x5000 pixel image files that you chose to chunk 125x125, for each image file your base table will have (5000*5000)/(125*125) records = 1600, your (l=2) o_2 table will have ceiling(1600/Power(2,2)) = 400 rows, your (l=3) o_3 will have ceiling(1600/Power(2,3) ) = 200 rows. If your pixels aren't divisible by the size of your tiles, you'll get some scrap tiles (tiles not completely filled). Note that each overview tile generated by raster2pgsql has the same number of pixels as its parent, but is of a lower resolution where each pixel of it represents (Power(2,overview_factor) pixels of the original).


An overview of the Raster Catalog toolset

A raster catalog is a collection of raster datasets. It is organized into rows each row contains a raster dataset, which is called a raster catalog item. The Raster Catalog toolset allows you to copy, create, edit, and delete raster catalogs and their items.

The following table lists the tools available in the Raster Catalog toolset and provides a brief description of each. The output for all these tools is a raster catalog.

Makes a copy of a raster catalog, including all of its contents, or a subset of its contents if there is a selection.

Creates an empty raster catalog in a geodatabase.

Deletes raster catalog items, including all its contents, or a subset of its contents if there is a selection.

Creates a table listing the paths to the raster datasets contained in an unmanaged raster catalog or a mosaic dataset. The table can display all the file paths, or just the ones that are broken.

Repairs broken file paths or deletes broken links within an unmanaged raster catalog or a mosaic dataset.

Loads all the raster datasets stored in the same workspace into an existing raster catalog.


An overview of the Raster Processing toolset

The Raster Processing toolset is used to perform some tasks to get your data to a workable form. The raster processing tools allow you to perform several tasks with your data to prepare it for analysis and display.

The following table lists the tools available in the Raster Processing toolset and provides a brief description of each. The input and output for all these tools is a raster dataset.

Cuts out a portion of a raster dataset, mosaic dataset, or image service layer.

Creates a single raster dataset from multiple bands.

Incorporates elevation data and image metadata to accurately line up imagery.

Fuses a high-resolution panchromatic raster dataset with a lower-resolution multiband raster dataset to create a red-green-blue (RGB) raster with the resolution of the panchromatic raster.

Calculates an optimal set of pan-sharpened weights for new or custom sensor data.

Creates a new raster dataset from a selection of an HDF or NITF dataset.

Splits a raster dataset into separate files based on the DTED tiling structure.

Change the spatial resolution of your raster dataset and set rules for aggregating or interpolating values across the new pixel sizes.

Divides a raster dataset into smaller pieces, by tiles or features from a polygon.


Parameters

The destination for the new raster datasets.

The prefix for each of the raster datasets you will create. A number will be appended to each prefix, starting with 0.

Determines how to split the raster dataset.

  • Size of tile — Specify the width and height of the tile.
  • Number of tiles — Specify the number of raster tiles to create by breaking the dataset into a number of columns and rows.
  • Polygon features — Use the individual polygon geometries in a feature class to split the raster.

The format for the output raster datasets.

  • Geotiff (*.tif) — Tagged Image File Format. This is the default.
  • Bitmap (*.bmp) — Microsoft Bitmap.
  • ENVI (*.dat) — ENVI DAT.
  • Esri BIL (*.bil) — Esri Band Interleaved by Line.
  • Esri BIP (*.bip) — Esri Band Interleaved by Pixel.
  • Esri BSQ (*.bsq) — Esri Band Sequential.
  • GIF (*.gif) — Graphic Interchange Format.
  • Esri GRID — Esri Grid.
  • ERDAS IMAGINE (*.img) — ERDAS IMAGINE.
  • JPEG 2000 (*.jp2) — JPEG 2000.
  • JPEG (*.jpeg) — Joint Photographic Experts Group.
  • PNG (*.png) — Portable Network Graphics.

Choose an appropriate technique based on the type of data you have.

  • Nearest — The fastest resampling method, and it minimizes changes to pixel values. Suitable for discrete data, such as land cover.
  • Bilinear — Calculates the value of each pixel by averaging (weighted for distance) the values of the surrounding 4 pixels. Suitable for continuous data.
  • Cubic — Calculates the value of each pixel by fitting a smooth curve based on the surrounding 16 pixels. Produces the smoothest image, but can create values outside of the range found in the source data. Suitable for continuous data.

The number of columns (x) and rows (y) to split the raster dataset into. The X coordinate is the number of columns and the Y coordinate is number of rows.

The x and y dimensions of the output tiles. The default unit of measurement is in pixels. You can change this with the Units for Output Raster Size and Overlap parameter. The X coordinate is the X (horizontal) dimension the output tiles and the Y coordinate is the Y (vertical) dimension of output tiles.

The tiles do not have to line up perfectly set the amount of overlap between tiles with this parameter. The default unit of measurement is in pixels. You can change this with the Units for Output Raster Size and Overlap parameter.

Set the units of measurement for the tile_size and the overlap parameters.

  • Pixels — The unit is in pixels. This is the default.
  • Meters — The unit is in meters.
  • Feet — The unit is in feet.
  • Degrees — The unit is in decimal degrees.
  • Miles — The unit is in miles.
  • Kilometers — The unit is in kilometers.

The spatial resolution of the output raster. If left blank, the output cell size will match the input raster. When you change the cell size values, the tile size is reset to the image size and the tile count is reset to 1.

Change the coordinates for the lower left origin point, where the tiling scheme will begin. If left blank, the lower left origin would be the same as the input raster.

A feature class that will be used to split the raster dataset.

Limits the extent of your raster dataset before you split it.

  • None — Use the full extent of the input raster dataset.
  • Extent —Specify bounding box as your clipping boundary.
  • Feature class —Specify a feature class to clip the extent.

An extent or a dataset used to define the clipping boundary. The dataset can be a raster or feature class.

  • Default —The extent will be based on the maximum extent of all participating inputs. This is the default.
  • Current Display Extent —The extent is equal to the data frame or visible display. The option is not available when there is no active map.
  • As Specified Below —The extent will be based on the minimum and maximum extent values specified.
  • Browse —The extent will be based on an existing dataset.

All the pixels with the specified value will be set to NoData in the output raster dataset.

Derived Output

The destination for the new raster datasets.

The prefix for each of the raster datasets you will create. A number will be appended to each prefix, starting with 0.

Determines how to split the raster dataset.

  • SIZE_OF_TILE — Specify the width and height of the tile.
  • NUMBER_OF_TILES — Specify the number of raster tiles to create by breaking the dataset into a number of columns and rows.
  • POLYGON_FEATURES — Use the individual polygon geometries in a feature class to split the raster.

The format for the output raster datasets.

  • TIFF — Tagged Image File Format. This is the default.
  • BMP — Microsoft Bitmap.
  • ENVI — ENVI DAT.
  • Esri BIL — Esri Band Interleaved by Line.
  • Esri BIP — Esri Band Interleaved by Pixel.
  • Esri BSQ — Esri Band Sequential.
  • GIF — Graphic Interchange Format.
  • GRID — Esri Grid.
  • IMAGINE IMAGE — ERDAS IMAGINE.
  • JP2 — JPEG 2000.
  • JPEG — Joint Photographic Experts Group.
  • PNG — Portable Network Graphics.

Choose an appropriate technique based on the type of data you have.

  • NEAREST — The fastest resampling method, and it minimizes changes to pixel values. Suitable for discrete data, such as land cover.
  • BILINEAR — Calculates the value of each pixel by averaging (weighted for distance) the values of the surrounding 4 pixels. Suitable for continuous data.
  • CUBIC — Calculates the value of each pixel by fitting a smooth curve based on the surrounding 16 pixels. Produces the smoothest image, but can create values outside of the range found in the source data. Suitable for continuous data.

The number of columns (x) and rows (y) to split the raster dataset into. This is a point whose X and Y coordinates define number of rows and columns. The X coordinate is the number of columns and the Y coordinate is the number of rows.

The x and y dimensions of the output tiles. The default unit of measurement is in pixels. You can change this with the units parameter. This is a point whose X and Y coordinates define the dimensions of output tiles. The X coordinate is the horizontal dimension of the output and the Y coordinate is the vertical dimension of the output.

The tiles do not have to line up perfectly set the amount of overlap between tiles with this parameter. The default unit of measurement is in pixels. You can change this with the units parameter.

Set the units of measurement for the tile_size and the overlap parameters.

  • PIXELS — The unit is in pixels. This is the default.
  • METERS — The unit is in meters.
  • FEET — The unit is in feet.
  • DEGREES — The unit is in decimal degrees.
  • MILES — The unit is in miles.
  • KILOMETERS — The unit is in kilometers.

The spatial resolution of the output raster. If left blank, the output cell size will match the input raster. When you change the cell size values, the tile size is reset to the image size and the tile count is reset to 1.

Change the coordinates for the lower left origin point, where the tiling scheme will begin. If left blank, the lower left origin would be the same as the input raster.

A feature class that will be used to split the raster dataset.

Limits the extent of your raster dataset before you split it.

  • NONE — Use the full extent of the input raster dataset.
  • EXTENT — Specify bounding box as your clipping boundary.
  • FEATURE_CLASS — Specify a feature class to clip the extent.

An extent or a dataset used to define the clipping boundary. The dataset can be a raster or feature class.

  • MAXOF —The maximum extent of all inputs will be used.
  • MINOF —The minimum area common to all inputs will be used.
  • DISPLAY —The extent is equal to the visible display.
  • Layer name—The extent of the specified layer will be used.
  • Extent object—The extent of the specified object will be used.
  • Space delimited string of coordinates—The extent of the specified string will be used. Coordinates are expressed in the order of x-min, y-min, x-max, y-max.

All the pixels with the specified value will be set to NoData in the output raster dataset.


ArcGIS Desktop direct connect and feature service access to supported DBMS content

ArcGIS feature services integrate data from multiple enterprise business systems.

  • ArcGIS Desktop software can connect to supported database (DBMS) content for view, query, and analysis operations.
  • ArcGIS Server can connect to supported databases and share tabular data as feature services.
  • ArcGIS Desktop supports remote editing by creating a local synchronized copy of the data from a published ArcGIS Server feature service.
  • Feature services can be registered with portal creating feature layer content for display and analysis in Web maps.
  • Web maps can be shared with authorized users throughout the organization through portal groups and displayed by a variety of mobile and Web-based COTS applications.
  • Web GIS client access for Branch versioning workflows are supported by registered feature services.

Enterprise geodatabases, also known as multiuser geodatabases, are stored in a relational database using DB2, Informix, Azure SQL Database, SQL Server, Oracle, PostgreSQL and SAP HANA. Amazon RDS for Microsoft SQL server, Amazon RDS for PostgreSQL, and Microsoft Azure SQL Server also support a cloud based Enterprise Geodatabase. Database technologies that do not support a geodatabase can still be leveraged for accessing spatial related data by the use of Query Layers. DBMS’s that support Query Layers include ALTIBASE, Dameng, Netezza, SQLLite, and Teradata.

Portal distributed collaboration

Portal collaboration plays a key role in sharing content between ArcGIS Enterprise organizations.

Content is shared to or from a group linked to a collaboration workspace.

The Host Portal admin defines how Guest Portals can access each collaboration workspace.

  • Send - content sent to the collaboration
  • Receive - content received from the collaboration
  • Send and receive – content sent to and received from the collaboration

ArcGIS 10.5.1+ sharing feature layer copies. (ArcGIS Online Host feature layer options)
ArcGIS 10.6+ provides feature layer copies between Portal for ArcGIS on-premise organizations.

  • ArcGIS 10.5.1+ supports hosted feature layers between ArcGIS Enterprise and ArcGIS Online organizations.
  • ArcGIS 10.5.1+ supports hosted feature layers between ArcGIS Enterprise organization.
  • Hosted feature layers are registered with portal published from a federated ArcGIS Enterprise server.

Portal collaboration deployment patterns

A variety of portal collaboration deployment patterns are supported with the ArcGIS 10.5.1 release.

Example deployment patterns

  • Centralized HUB pattern.
  • Collaboration between organizations.
  • Integration between an ArcGIS Online and ArcGIS Enterprise organizations.

Distributed collaboration options include synchronizing Portal group content and copy/synchronizing Portal group feature layer content between ArcGIS Enterprise and ArcGIS Online organizations. The option to copy feature layer content between ArcGIS Enterprise organizations will be supported with the ArcGIS 10.6 release.

ArcGIS Data Store

Base ArcGIS Enterprise configuration

  • Portal for ArcGIS
  • Federated GIS Server (configured as hosting server)
  • ArcGIS Data Stores (accessed through hosting server)

Enterprise Geodatabase can used as a managed data source for published services registered with Portal.

ArcGIS Data Stores configurations

ArcGIS Data Store utilization

  • Relational Data Store. Portal feature services, ArcGIS Image Server, and ArcGIS GeoAnalytics Server.
  • Tile Cache Data Store. Portal 3D Scenes and ArcGIS Image Server.
  • Spatiotemporal Big Data Store. GeoEvent Server and ArcGIS GeoAnalytics Server.

Base ArcGIS Enterprise configuration is required for ArcGIS Data Store deployment.


Loop to extract specific raster/spatial point pairs

I have many point shapefiles which are not overlapping, which I would like to attribute to analagous rasters, also not overlapping. I would like to attribute these points with the raster data. For some of the raster data types with which I am doing this, I was able to merge the rasters first, then attribute. However, my last sets of raster data do not have the same origin, so I was unable to merge/mosaic them. I am trying to instead attribute the points to rasters without merging the rasters. This would require me to use extract() on specific spatial point - raster pairs. I have named each spatial point file with a unique 4-letter name, which is also part of the raster name I would like the extract() to use.

I've created a reproducible example below that mimics my data and problem. Can anyone make suggestions on how I could code the looping for extract() to get the spatial point file to extract to the analagously-named raster?

Alternatively, if it is better/possible to combine all the spatial points and just loop through extracting all the rasters, then managing the data so that all extracted values are in one vector or column of a dataframe, perhaps that would be better.

I am using RStudio 1.2.1335

Note: I posted this question to GIS Stack Exchange, but did not receive any answers, hope cross-posting is ok.


GIS enterprise data architecture

Figure 5.25 shows the GIS enterprise data architecture. GIS enterprise architecture often includes both feature and imagery data within the data center, requiring effective data management and automation to maintain the variety of data sources. The GIS data administrator must manage a hybrid architecture containing a mix of resources stored on file systems and multiple database platforms. ArcGIS technology provides a variety of processing and replication functions for maintaining data resources in an optimum configuration.

Your optimum data configuration will depend on your business needs.


ImageryLayer¶

The attribute_table method returns categorical mapping of pixel values (for example, a class, group, category, or membership).

Description

Specifies the rendering rule for how the requested image should be processed. The response is updated Layer info that reflects a custom processing as defined by the rendering rule. For example, if renderingRule contains an attributeTable function, the response will indicate “hasRasterAttributeTable”: true if the renderingRule contains functions that alter the number of bands, the response will indicate a correct bandCount value.

returns the band count of the imagery layer

overlapping pixels at the same location are resolved by blending all overlapping pixels

this imagery layer with mosaic operation set to ‘blend’

Provides access to the tools to update, add, and remove cache on the ImageLayer

ImageryLayerCacheManager or None

calculate_volume ( geometries , base_type = None , mosaic_rule = None , constant_z = None , pixel_size = None ) ¶

Performs volumetric calculation on an elevation service. Results are always in square meters (area) and cubic meters (volume). If a service does not have vertical spatial reference and z unit is not in meters, user needs to apply a conversion factor when interpreting results.

Available in 10.7+ only

Description

required a list of Polygon geometry objects or a list of envelope geometry objects. A geometry that defines the geometry within which the volume is computed. The geometry can be an envelope or a polygon

optional integer. 0 - constant z 1 - best fitting plane 2 - lowest elevation on the perimeter 3 - highest elevation on the perimeter 4 - average elevation on the perimeter

Optional dictionary. Used to select different DEMs in a mosaic dataset

Optional integer. parameter to specify constant z value

Optional string or dictionary. Defines the spatial resolution at which volume calculation is performed Syntax:

  • dictionary structure: pixel_size=

  • Point simple syntax: pixel_size=’<x>,<y>’

dictionary showing volume values for each geometry in the input geometries array

The Raster Catalog Item property represents a single raster catalog item

Description

required integer. The id is the ‘raster id’.

The colormap method returns RGB color representation of pixel values. This method is supported if the hasColormap property of the layer is true.

Description

optional dictionary. Specifies the rendering rule for how the requested image should be rendered. See the raster function objects for the JSON syntax and examples. https://developers.arcgis.com/documentation/common-data-types/raster-function-objects.htm

Optional String. This parameter can be used to request a colormap for each variable for an image service that has multidimensional information. It will return a colormap for the whole image service if not specified. Eligible variable names can be queried from multidimensional_info property of the Imagery Layer object. This parameter is available from 10.8.1

returns number of columns in the imagery layer

compute_class_stats ( descriptions , mosaic_rule = 'defaultMosaicMethod' , rendering_rule = None , pixel_size = None ) ¶

Compute class statistics signatures (used by the maximum likelihood classifier)

Description

Required list. Class descriptions are training site polygons and their class descriptions. The structure of the geometry is the same as the structure of the JSON geometry objects returned by the ArcGIS REST API.

optional string. Specifies the mosaic rule when defining how individual images should be mosaicked. When a mosaic rule is not specified, the default mosaic rule of the image layer will be used (as advertised in the root resource: defaultMosaicMethod, mosaicOperator, sortField, sortValue). See Mosaic rule objects help for more information: https://developers.arcgis.com/documentation/common-data-types/mosaic-rules.htm

optional dictionary. Specifies the rendering rule for how the requested image should be rendered. See the raster function objects for the JSON syntax and examples. https://developers.arcgis.com/documentation/common-data-types/raster-function-objects.htm

optional list or dictionary. The pixel level being used (or the resolution being looked at). If pixel size is not specified, then pixel_size will default to the base resolution of the dataset. The structure of the pixel_size parameter is the same as the structure of the point object returned by the ArcGIS REST API. In addition to the dictionary structure, you can specify the pixel size with a comma-separated syntax.

  • dictionary structure: pixel_size=

  • Point simple syntax: pixel_size=’<x>,<y>’

  • pixel_size=

  • pixel_size=’0.18,0.18’

compute_histograms ( geometry , mosaic_rule = None , rendering_rule = None , pixel_size = None , time = None , process_as_multidimensional = False ) ¶

The compute_histograms operation is performed on an imagery layer method. This operation is supported by any imagery layer published with mosaic datasets or a raster dataset. The result of this operation contains both statistics and histograms computed from the given extent.

Description

required Polygon or Extent. A geometry that defines the geometry within which the histogram is computed. The geometry can be an envelope or a polygon

optional string. Specifies the mosaic rule when defining how individual images should be mosaicked. When a mosaic rule is not specified, the default mosaic rule of the image layer will be used (as advertised in the root resource: defaultMosaicMethod, mosaicOperator, sortField, sortValue). See Mosaic rule objects help for more information: https://developers.arcgis.com/documentation/common-data-types/mosaic-rules.htm

Specifies the rendering rule for how the requested image should be processed. The response is updated Layer info that reflects a custom processing as defined by the rendering rule. For example, if renderingRule contains an attributeTable function, the response will indicate “hasRasterAttributeTable”: true if the renderingRule contains functions that alter the number of bands, the response will indicate a correct bandCount value.

optional list or dictionary. The pixel level being used (or the resolution being looked at). If pixel size is not specified, then pixel_size will default to the base resolution of the dataset. The structure of the pixel_size parameter is the same as the structure of the point object returned by the ArcGIS REST API. In addition to the dictionary structure, you can specify the pixel size with a comma-separated string.

dictionary structure: pixel_size=

Point simple syntax: pixel_size=’<x>,<y>’

optional datetime.date, datetime.datetime or timestamp string. The time instant or the time extent of the exported image. Time instant specified as datetime.date, datetime.datetime or timestamp in milliseconds since epoch Syntax: time=<timeInstant>

Time extent specified as list of [<startTime>, <endTime>] For time extents one of <startTime> or <endTime> could be None. A None value specified for start time or end time will represent infinity for start or end time respectively. Syntax: time=[<startTime>, <endTime>] specified as datetime.date, datetime.datetime or timestamp

optional boolean. Specifies whether to process the image service as a multidimensional image service.

  • False - The histogram of pixel values from only the first slice is computed. This is the default.

  • True - The image service is treated as a multidimensional raster, and histograms of pixel values from all selected slices are computed.

Added at 10.9 for image services which use ArcObjects11 or ArcObjectsRasterRendering as the service provider.

With given input geometries, it calculates corresponding pixel location in column and row on specific raster catalog item. A prerequisite is that the raster catalog item has valid icsToPixel resource.

Description

required integer. Specifies the objectId of image service’s raster catalog. This integer rasterId number will determine which raster’s image coordinate system will be used during the calculation and which raster does the column and row of results represent.

The array of geometries for computing pixel locations. All geometries in this array should be of the type defined by geometryType.

required string, dictionary, This specifies the spatial reference of the Geometries parameter above. It can accept a multitudes of values. These can be a WKID, image coordinate system (ICSID), or image coordinate system in json/dict format. Additionally the arcgis.geometry.SpatialReference object is also a valid entry. .. note :: An image coordinate system ID can be specified using 0:icsid for example, 0:64. The extra 0: is used to avoid conflicts with wkid

dictionary, The result of this operation includes x and y values for the column and row of each input geometry. It also includes a z value for the height at given location based on elevation info that the catalog raster item has.

compute_stats_and_histograms ( geometry , mosaic_rule = None , rendering_rule = None , pixel_size = None , time = None , process_as_multidimensional = False ) ¶

The result of this operation contains both statistics and histograms computed from the given extent.

Description

required Polygon or Extent. A geometry that defines the geometry within which the histogram is computed. The geometry can be an envelope or a polygon

optional dictionary. Specifies the mosaic rule when defining how individual images should be mosaicked. When a mosaic rule is not specified, the default mosaic rule of the image layer will be used (as advertised in the root resource: defaultMosaicMethod, mosaicOperator, sortField, sortValue).

optional dictionary. Specifies the rendering rule for how the requested image should be rendered.

optional string or dict. The pixel level being used (or the resolution being looked at). If pixel size is not specified, then pixel_size will default to the base resolution of the dataset. The raster at the specified pixel size in the mosaic dataset will be used for histogram calculation.

dictionary structure: pixel_size=

Point simple syntax: pixel_size=’<x>,<y>’

optional datetime.date, datetime.datetime or timestamp string. The time instant or the time extent of the exported image. Time instant specified as datetime.date, datetime.datetime or timestamp in milliseconds since epoch Syntax: time=<timeInstant>

Time extent specified as list of [<startTime>, <endTime>] For time extents one of <startTime> or <endTime> could be None. A None value specified for start time or end time will represent infinity for start or end time respectively. Syntax: time=[<startTime>, <endTime>] specified as datetime.date, datetime.datetime or timestamp

optional boolean. Specifies whether to process the image service as a multidimensional image service.

  • False - Statistics and histograms of pixel values from only the first slice is computed. This is the default.

  • True - The image service is treated as a multidimensional raster, and statistics and histograms of pixel values from all selected slices are computed.

Added at 10.9 for image services which use ArcObjects11 or ArcObjectsRasterRendering as the service provider.

The result of this operation contains tie points that can be used to match the source image to the reference image. The reference image is configured by the image layer publisher. For more information, see Fundamentals for georeferencing a raster dataset.

Description

required integer. Source raster ID.

required dictionary. The geodata transformation that provides a rough fit of the source image to the reference image. For example, a first order polynomial transformation that fits the source image to the expected location.

draw_graph ( show_attributes = False , graph_size = '14.25, 15.25' ) ¶

Displays a structural representation of the function chain and it’s raster input values. If show_attributes is set to True, then the draw_graph function also displays the attributes of all the functions in the function chain, representing the rasters in a blue rectangular box, attributes in green rectangular box and the raster function names in yellow.

Description

optional boolean. If True, the graph displayed includes all the attributes of the function and not only it’s function name and raster inputs Set to False by default, to display only he raster function name and the raster inputs to it.

optional string. Maximum width and height of drawing, in inches, seperated by a comma. If only a single number is given, this is used for both the width and the height. If defined and the drawing is larger than the given size, the drawing is uniformly scaled down so that it fits within the given size.

export_image ( bbox = None , image_sr = None , bbox_sr = None , size = None , time = None , export_format = 'jpgpng' , pixel_type = None , no_data = None , no_data_interpretation = 'esriNoDataMatchAny' , interpolation = None , compression = None , compression_quality = None , band_ids = None , mosaic_rule = None , rendering_rule = None , f = 'json' , save_folder = None , save_file = None , compression_tolerance = None , adjust_aspect_ratio = None , lerc_version = None , slice_id = None ) ¶

The export_image operation is performed on an imagery layer. The result of this operation is an image method. This method provides information about the exported image, such as its URL, extent, width, and height. In addition to the usual response formats of HTML and JSON, you can also request the image format while performing this operation. When you perform an export with the image format , the server responds by directly streaming the image bytes to the client. With this approach, you don’t get any information associated with the exported image other than the image itself.

Description

Optional dict or string. The extent (bounding box) of the exported image. Unless the bbox_sr parameter has been specified, the bbox is assumed to be in the spatial reference of the imagery layer.

The bbox should be specified as an arcgis.geometry.Envelope object, it’s json representation or as a list or string with this format: ‘<xmin>, <ymin>, <xmax>, <ymax>’ If omitted, the extent of the imagery layer is used

optional string, SpatialReference. The spatial reference of the exported image. The spatial reference can be specified as either a well-known ID, it’s json representation or as an arcgis.geometry.SpatialReference object. If the image_sr is not specified, the image will be exported in the spatial reference of the imagery layer.

optional string, SpatialReference. The spatial reference of the bbox. The spatial reference can be specified as either a well-known ID, it’s json representation or as an arcgis.geometry.SpatialReference object. If the image_sr is not specified, bbox is assumed to be in the spatial reference of the imagery layer.

optional list. The size (width * height) of the exported image in pixels. If size is not specified, an image with a default size of 1200*450 will be exported. Syntax: list of [width, height]

optional datetime.date, datetime.datetime or timestamp string. The time instant or the time extent of the exported image. Time instant specified as datetime.date, datetime.datetime or timestamp in milliseconds since epoch Syntax: time=<timeInstant>

Time extent specified as list of [<startTime>, <endTime>] For time extents one of <startTime> or <endTime> could be None. A None value specified for start time or end time will represent infinity for start or end time respectively. Syntax: time=[<startTime>, <endTime>] specified as datetime.date, datetime.datetime or timestamp

optional string. The format of the exported image. The default format is jpgpng. The jpgpng format returns a JPG if there are no transparent pixels in the requested extent otherwise, it returns a PNG (png32).

optional string. The pixel type, also known as data type, pertains to the type of values stored in the raster, such as signed integer, unsigned integer, or floating point. Integers are whole numbers, whereas floating points have decimals.

optional float. The pixel value representing no information.

optional string. Interpretation of the no_data setting. The default is NoDataMatchAny when no_data is a number, and NoDataMatchAll when no_data is a comma-delimited string: NoDataMatchAny,NoDataMatchAll.

optional string. The resampling process of extrapolating the pixel values while transforming the raster dataset when it undergoes warping or when it changes coordinate space. One of: RSP_BilinearInterpolation, RSP_CubicConvolution, RSP_Majority, RSP_NearestNeighbor

optional string. Controls how to compress the image when exporting to TIFF format: None, JPEG, LZ77. It does not control compression on other formats.

optional integer. Controls how much loss the image will be subjected to by the compression algorithm. Valid value ranges of compression quality are from 0 to 100.

optional list. If there are multiple bands, you can specify a single band to export, or you can change the band combination (red, green, blue) by specifying the band number. Band number is 0 based. Specified as list of ints, eg [2,1,0]

optional dict. Specifies the mosaic rule when defining how individual images should be mosaicked. When a mosaic rule is not specified, the default mosaic rule of the image layer will be used (as advertised in the root resource: defaultMosaicMethod, mosaicOperator, sortField, sortValue).

optional dict. Specifies the rendering rule for how the requested image should be rendered.

optional string. The response format. default is json Values: json,image,kmz If image format is chosen, the bytes of the exported image are returned unless save_folder and save_file parameters are also passed, in which case the image is written to the specified file

optional string. The folder in which the exported image is saved when f=image

optional string. The file in which the exported image is saved when f=image

optional float. Controls the tolerance of the lerc compression algorithm. The tolerance defines the maximum possible error of pixel values in the compressed image. Example: compression_tolerance=0.5 is loseless for 8 and 16 bit images, but has an accuracy of +-0.5 for floating point data. The compression tolerance works for the LERC format only.

optional boolean. Indicates whether to adjust the aspect ratio or not. By default adjust_aspect_ratio is true, that means the actual bbox will be adjusted to match the width/height ratio of size paramter, and the response image has square pixels.

optional integer. The version of the Lerc format if the user sets the format as lerc. Values: 1 or 2 If a version is specified, the server returns the matching version, or otherwise the highest version available.

optional integer. Exports the given slice of a multidimensional raster. To get the slice index use slices method on the ImageryLayer object.

Area of interest. Used for displaying the imagery layer when queried

filter_by ( where = None , geometry = None , time = None , lock_rasters = True ) ¶

Filters the layer by where clause, geometry and temporal filters

Description

optional string. A where clause on this layer to filter the imagery layer by the selection sql statement. Any legal SQL where clause operating on the fields in the raster

optional arcgis.geometry.filters. To filter results by a spatial relationship with another geometry

optional datetime, date, or timestamp. A temporal filter to this layer to filter the imagery layer by time using the specified time instant or the time extent.

Time extent specified as list of [<startTime>, <endTime>] For time extents one of <startTime> or <endTime> could be None. A None value specified for start time or end time will represent infinity for start or end time respectively. Syntax: time_filter=[<startTime>, <endTime>] specified as datetime.date, datetime.datetime or timestamp in milliseconds

optional boolean. If True, the LockRaster mosaic rule will be applied to the layer, unless overridden

ImageryLayer with filtered images meeting the filter criteria

The object ids of the filtered rasters in this imagery layer, by applying the where clause, spatial and temporal filters. If no rasters are filtered, returns None. If all rasters are filtered, returns empty list

overlapping pixels at the same location are resolved by picking the first image :return: this imagery layer with mosaic operation set to ‘first’

classmethod fromitem ( item ) ¶

Returns the layer at the specified index from a layer item.

Description

Required string. An item ID representing a layer.

Optional int. The index of the layer amongst the item’s layers

The layer at the specified index.

get_download_info ( raster_ids , polygon = None , extent = None , out_format = None ) ¶

The Download Rasters operation returns information (the file ID) that can be used to download the raw raster files that are associated with a specified set of rasters in the raster catalog.

Description

required string. A comma-separated list of raster IDs whose files are to be downloaded.

optional Polygon, The geometry to apply for clipping

optional string. The geometry to apply for clipping example: “-104,35.6,-94.32,41”

optional string. The format of the rasters returned. If not specified, the rasters will be in their native format. The format applies when the clip geometry is also specified, and the format will be honored only when the raster is clipped.

To force the Download Rasters operation to convert source images to a different format, append :Conversion after format string. Valid formats include: TIFF, Imagine Image, JPEG, BIL, BSQ, BIP, ENVI, JP2, GIF, BMP, and PNG. Example: out_format=’TIFF’

get_histograms() can return histogram for each variable if used with multidimensional ImageryLayer object by specifing value for variable parameter.

If histogram is not found, returns None. In this case, call the compute_histograms(). (get_histograms() is an enhanced version of the histograms property on the ImageryLayer class with additional variable parameter.)

Description

Optional string. For an image service that has multidimensional information, this parameter can be used to request histograms for each variable. It will return histograms for the whole ImageryLayer if not specified. This parameter is available from 10.8.1

#length of this list corresponds ‘size’

get_raster_file ( download_info , out_folder = None ) ¶

The Raster File method represents a single raw raster file. The download_info is obtained by using the get_download_info operation.

Description

required dictionary. This is derived from the get_downlad_info().

optional string. Path to the file save location. If the value is None, the OS temporary directory is used.

get_samples ( geometry , geometry_type = None , sample_distance = None , sample_count = None , mosaic_rule = None , pixel_size = None , return_first_value_only = None , interpolation = None , out_fields = None ) ¶

The get_samples operation is supported by both mosaic dataset and raster dataset imagery layers. The result of this operation includes sample point locations, pixel values, and corresponding spatial resolutions of the source data for a given geometry. When the input geometry is a polyline, envelope, or polygon, sampling is based on sample_count or sample_distance when the input geometry is a point or multipoint, the point or points are used directly. The number of sample locations in the response is based on the sample_distance or sample_count parameter and cannot exceed the limit of the image layer (the default is 1000, which is an approximate limit).

Description

A geometry that defines the location(s) to be sampled. The structure of the geometry is the same as the structure of the JSON geometry objects returned by the ArcGIS REST API. Applicable geometry types are point, multipoint, polyline, polygon, and envelope. When spatial reference is omitted in the input geometry, it will be assumed to be the spatial reference of the image layer.

optional string. The type of geometry specified by the geometry parameter. The geometry type can be point, multipoint, polyline, polygon, or envelope.

optional float. The distance interval used to sample points from the provided path. The unit is the same as the input geometry. If neither sample_count nor sample_distance is provided, no densification can be done for paths (polylines), and a default sample_count (100) is used for areas (polygons or envelopes).

optional integer. The approximate number of sample locations from the provided path. If neither sample_count nor sample_distance is provided, no densification can be done for paths (polylines), and a default sample_count (100) is used for areas (polygons or envelopes).

optional dictionary. Specifies the mosaic rule when defining how individual images should be mosaicked. When a mosaic rule is not specified, the default mosaic rule of the image layer will be used (as advertised in the root resource: defaultMosaicMethod, mosaicOperator, sortField, sortValue).

optional string or dict. The pixel level being used (or the resolution being looked at). If pixel size is not specified, then pixel_size will default to the base resolution of the dataset. The raster at the specified pixel size in the mosaic dataset will be used for histogram calculation.

dictionary structure: pixel_size=

Point simple syntax: pixel_size=’<x>,<y>’

optional boolean. Indicates whether to return all values at a point, or return the first non-NoData value based on the current mosaic rule. The default is true.

optional string. The resampling method. Default is nearest neighbor. Values: RSP_BilinearInterpolation,RSP_CubicConvolution,

optional string. The list of fields to be included in the response. This list is a comma-delimited list of field names. You can also specify the wildcard character (*) as the value of this parameter to include all the field values in the results.

returns height of image service

Returns the histograms of each band in the imagery layer as a list of dictionaries corresponding to each band. If not histograms is found, returns None. In this case, call the compute_histograms()


4 Answers 4

Information retrieval is based on a query - you specify what information you need and it is returned in human understandable form.

Information extraction is about structuring unstructured information - given some sources all of the (relevant) information is structured in a form that will be easy for processing. This will not necessary be in human understandable form - it can be only for use of computer programs.

http://gate.ac.uk/ie/ gives a very nice, concise distinction:

Information Extraction is not Information Retrieval: Information Extraction differs from traditional techniques in that it does not recover from a collection a subset of documents which are hopefully relevant to a query, based on key-word searching (perhaps augmented by a thesaurus). Instead, the goal is to extract from the documents (which may be in a variety of languages) salient facts about prespecified types of events, entities or relationships. These facts are then usually entered automatically into a database, which may then be used to analyse the data for trends, to give a natural language summary, or simply to serve for on-line access.

Information Retrieval gets sets of relevant documents:

Information Extraction gets facts out of documents:

From a modeling standpoint, information retrieval is a deep field predicated on several disciplines, including statistics, math, linguistics, artificial intelligence and now data science. In practice, these models are applied against text within corpora to discover patterns in the data. Not only do the IR models overlap in their usage, they can "partner" with other models such as k-means or k-nearest neighbor models, then other models can be applied from the vantage point of computational linguistics such as LDA/LDI and topic modeling Then, the end game is some sort of information visualization of this discovery--after ranking, clustering and aggregating work. Information Retrieval may appear to be a cryptic discipline, but serious effort, which is greatly appreciated, is going into opening up the area for deeper understanding of each model, and the interaction between models. I cite "Synthesis Lectures on Information Concepts, Retrieval, and Services" Series as the best place to delve into a foundation for IR.

While I don't entirely separate IR and Information Extraction, perhaps a subset of IE, concept level extraction, does apply IR patterns along with AI-based inferencing rules to extract related ontologies. The graphical nature of these relations are being enhanced with ontology modeling in OWL and RDF, and with graph databases, which allow for a less strict or rigorous set of relationship modeling, and allow for more relationships to surface, rather than being controlled per se. The ability to grow information extraction dynamically keeps its "discipline" strongly interesting to researchers.

Both IR and IE play out in our own significant "entities of the moment"--some have called "dynamic ontologies"--some being Palantir-- we need the patterns, models, simulations and visualizations of those significant entities to do business in the face of morphing new sources of information and changing of existing information. The conceptual, relational, definitional, pattern and ontological modeling have to be flexible and their visualizations the same. The heavy lifting of AI engines such as Watson in the information extraction and inferencing fields has cast a spotlight on the IE and frankly IR fields. Also the ubiquity of natural language processing and machine learning are calling attention to IR and IE models and engines. The impact of IR models on search and SEO, and on semantic web modeling is one of those "watching a change agent change due to impacts back from what it impacted" events--somewhere in harmonics and relativity theory. IR and IE are expanding based on what they're impacting.


Watch the video: ArcGIS - Mosaic Raster data - Mosaic DEM