ProSuite package

Data Model

class prosuite.data_model.BaseDataset(name: str, filter_expression: str = '')

Bases: object

The base class for datasets representing tabular data. It is either a Dataset or, in the future, a TransformedDataset

class prosuite.data_model.Dataset(name: str, model: Model, filter_expression: str = '')

Bases: BaseDataset

A dataset represents data from a table or feature class in a workspace, optionally filtered by an SQL expression.

Parameters

• name (str) – table or featureclass name

• model (class:prosuite.data_model.Model) – The prosuite.data_model.Model this dataset belongs to.

• filter_expression (str, optional) – A where clause that filters the table. The syntax of the where clause is defined in the document SQLSyntax_en.pdf

class prosuite.data_model.Model(name, catalog_path)

Bases: object

The Model represents the data model in a workspace (file-gdb or enterprise-gdb)

catalog_path examples:

c:/data.gdb c:/enterprise_gdb.sde

catalog_path: str

The catalog path of the associated workspace.

name: str

The unique name of the model.

class prosuite.data_model.TransformedDataset(transformer_descriptor: str, name: str = '', filter_expression: str = '')

Bases: BaseDataset

generate_name()

Generates a technical name using the dataset name(s) and the test descriptor. This is the default name of a condition if it was created by the standard factory method from prosuite.factories.quality_conditions.

name: str

The unique name of the transformed dataset.

parameters: List[Parameter]

The list of parameters. Typically the parameters are specified in the factory method used to create the transformed dataset (see prosuite.factories.transformers) and do not need to be changed through this list.

transformer_descriptor

The transformer descriptor, i.e. the algorithm used to generate this dataset.

prosuite.data_model.create_name(descriptor, params) → str

Quality

class prosuite.quality.Condition(test_descriptor: str, name: str = '')

Bases: object

Defines a quality condition, i.e. the configuration of a test algorithm for one or more datasets. Conditions must be created with the factory methods from prosuite.factories.quality_conditions

allow_errors: bool

For internal use only.

category: str

The name of the category, if this issue is assigned to a category.

description: str

Freely definable description of the Quality Condition. This description can be displayed when viewing issues in the issue navigator, and may contain explanations to the Quality Condition and instructions for correcting the issues.

generate_name()

Generates a technical name using the dataset name(s) and the test descriptor. This is the default name of a condition if it was created by the standard factory method from prosuite.factories.quality_conditions.

issue_filter_expression: str

Reserved for future use.

issue_filters: List[IssueFilter]

Reserved for future use.

issue_type: IssueType

Defines if a failing test returns a warning or an error issue. Quality conditions with Issue Type = Warning are considered “soft” conditions, for which exceptions (“Allowed Issues”) may be defined.

name

The unique name of the quality condition.

parameters: List[Parameter]

The list of parameters. Typically the parameters are specified in the factory method used to create the quality condition (see prosuite.factories.quality_conditions) and do not need to be changed through this list.

stop_on_error: bool

Indicates if the occurrence of an error for an object should stop any further testing of the same object. This can be used to prevent further tests on a feature after a serious geometry error (e.g. incorrectly oriented rings) was detected for the feature. The used Test Descriptor provides a standard value for this property. It can optionally be overridden here.

test_descriptor

The test descriptor, i.e. the algorithm used to verify this condition.

url: str

Optional URL to a website providing additional information for this Quality Condition. Certain Quality Conditions require more detailed information about the test logic and/or the correction guidelines than the field “Description” can provide. This information can for example be assembled in a wiki, and the URL may be provided here. When viewing issues in the issue navigator, the corresponding web page can be opened. In the HTML verification reports these URLs are used to render the names of the Quality Conditions as links.

class prosuite.quality.Parameter(name: str, value)

Bases: object

A parameter configures a quality condition. Parameters can represent Datasets (dataset parameter) or scalar values (scalar parameters). Parameters have a name and a value.

Dataset parameters: value is of type dataset. the parameter can retrieve the workspace id (model name) and the where clause (filter expression) of the dataset.

Scalar parameters: value is a simple type (number, string, bool).

get_string_value() → str

get_where_clause()

get_workspace_id()

is_dataset_parameter() → bool

static value_is_list_of_datasets(value)

class prosuite.quality.Specification(name: str = 'Custom Specification', description: str = '')

Bases: object

add_condition(condition: Condition)

Adds conditions to the specification

get_conditions() → List[Condition]

Returns the List of conditions

Verification

class prosuite.verification.AdvancedParameters(specification, output_dir, perimeter, verification_params: Optional[VerificationParameters] = None)

Bases: object

class prosuite.verification.EnvelopePerimeter(x_min: float, y_min: float, x_max: float, y_max: float)

Bases: object

A spatial envelope defined by the bounding coordinates. The spatial reference must match the spatial reference of the datasets.

class prosuite.verification.EsriShapePerimeter(esri_shape: bytes)

Bases: object

A polygon in the Esri shape buffer format.

class prosuite.verification.InvolvedTable(table_name: str, object_ids: list)

Bases: object

Represents a table involved in a verification issue.

object_ids

A list of object IDs from the table that are involved in the issue.

table_name

The name of the table.

class prosuite.verification.Issue(issue_msg: prosuite.generated.shared_qa_pb2.IssueMsg)

Bases: object

Represents an issue found during the verification.

To initialize an Issue object, pass the issue message to it. The Issue object extracts some properties from the issue message and makes them available as attributes:

allowable

If the issue is allowable.

description

The description of the issue.

geometry

The geometry involved of the issue.

involved_objects

A list of InvolvedTable objects that are involved in the issue.

issue_code

The issue code ID.

stop_condition

If the issue is a stop condition.

class prosuite.verification.MessageLevel

Bases: object

level_10000 = 'Verbose'

level_110000 = 'Fatal'

level_30000 = 'Debug'

level_40000 = 'Info'

level_60000 = 'Warn'

level_70000 = 'Error'

class prosuite.verification.Service(host_name: str, port_nr: int, channel_credentials: Optional[grpc.ssl_channel_credentials] = None)

Bases: object

The service class communicates on the http/2 channel with the server and initiates the quality verifications.

HTML_REPORT = 'verification.html'

The name of the html verification report. It will be written to the output_dir specified in the prosuite.service.Service.verify() method.

ISSUE_GDB = 'Issues.gdb'

The name of the issue File Geodatabase. It will be written to the output_dir specified in the prosuite.service.Service.verify() method. This File Geodatabase contains the issues found during the verification and could be used as the source for the Issue Worklist in the ProSuite QA Add-In for ArcGIS Pro.

XML_REPORT = 'verification.xml'

The name of the xml verification report. It will be written to the output_dir specified in the prosuite.service.Service.verify() method.

host_name

The name or IP address of the host running the quality verification service.

port_nr

The port used by the quality verification service.

ssl_channel_credentials: grpc.ssl_channel_credentials

The channel credentials to be used for TLS/SSL server authentication, if required by the server (Default: None -> No TLS/SSL).

Use prosuite.utils.get_ssl_channel_credentials() to create the basic https credentials if the appropria root certificates are in the windows certificate store. For advanced scenarios or credentials on a non-windows platform, see the gRPC Python docs (https://grpc.github.io/grpc/python/grpc.html).

verify(specification: Union[Specification, XmlSpecification, DdxSpecification], perimeter: Optional[Union[EnvelopePerimeter, EsriShapePerimeter, WkbPerimeter]] = None, output_dir: Optional[str] = None, parameters: Optional[VerificationParameters] = None) → Iterable[VerificationResponse]

Executes a quality verification by running all the quality conditions defined in the provided quality specification. Returns a collection of VerificationResponse objects, containing the verification messages.

Please refer to the samples for more details.

Parameters

• specification – The quality specification containing the conditions to be verified. It can be either a prosuite.quality.Specification directly defined in python code or a prosuite.quality.XmlSpecification from an XML file, for example created by the XML export in the ProSuite Data Dictionary Editor.

• perimeter – The perimeter that defines the polygon or extent of the verification. Default: None -> Full extent of the verified datasets.

• output_dir – The output directory (must be writable / creatable by the service process). Default: No output is written by the server process.

• parameters – Additional optional verification parameters.

Returns

Iterator for looping over VerificationResponse objects. The verification response contains progress information, found issues and, in the final message, the verification results.

Return type

Iterator[VerificationResponse]

class prosuite.verification.ServiceStatus

Bases: object

status_0 = 'Undefined'

status_1 = 'Running'

status_2 = 'Cancelled'

status_3 = 'Finished'

status_4 = 'Failed'

class prosuite.verification.VerificationParameters(tile_size: int = 5000, user_name: Optional[str] = None)

Bases: object

Contains all parameters that can be passed to a verification.

add_objects_to_verify(dataset_id: int, object_ids: list)

Adds a dataset and a list of object IDs to the objects to be verified.

Parameters

• dataset_id (int) – The dataset ID containing the selected datasets The id can be look up in the DataDictionaryEditor -> Data -> Data Models -> Dataset -> Properties.

• object_ids (list) – A list of feature-object IDs from the dataset to be verified.

desired_parallel_processing: int

The desired number of parallel worker processes to be used if the server allows parallel processing.

objects_to_verify

A dictionary containing the dataset IDs and the object IDs to be verified. The dataset ID can be looked up in the Data Dictionary Editor -> Data -> Data Models -> Dataset -> Properties. Use the method add_objects_to_verify to add a list of IDs for a specific dataset.

save_verification_statistics

If True, the verification statistics will be saved into the Data Dictionary database.

tile_size: int

The size (in meter) for testing quality conditions.

update_issues_in_verified_model

If True, the issues will be updated in the error datasets of the verified model.

user_name: str

The executing user which will be used in issue features.

class prosuite.verification.VerificationResponse(service_call_status: str, message: str, message_level: str, issue_msgs: list)

Bases: object

This class represents a VerificationResponse Message. The str() method is overridden to return all properties from the VerificationResponse when a VerificationResponse object is printed using pythons print() method.

issues

message: str

the actual message

message_level

message level -> see class MessageLevel

service_call_status

service status -> see class ServiceStatus

class prosuite.verification.WkbPerimeter(wkb: bytes)

Bases: object

A polygon in the OGC well-known-binary format. For example, in ArcPy the geometry’s WKB property could be used to acquire this format.

Utils

prosuite.utils.append_timestamp_to_basepath(base_path)

prosuite.utils.get_ssl_channel_credentials() → grpc.ssl_channel_credentials

Creates an ssl channel credentials object for use with an SSL-enabled channel for the authentication with a server that requires TLS/SSL. The appropriate root certificates for server authentication are loaded from the windows certificate store.

Returns

A ChannelCredentials object

prosuite.utils.get_value_or_default(value: Any, default: Any) → Any

Returns the value if it is not None. Else returns the default value.

prosuite.utils.load_file_as_string(file_name: str) → str

prosuite.utils.load_json_file(json_file_path: str)

prosuite.utils.objectify_xml(path_file) → lxml.objectify.ObjectifiedElement

Try to objectify the xml file and return the objectified element. Return None if it fails.

prosuite.utils.str_is_none_or_empty(string: str) → bool

prosuite.utils.str_not_empty(string: str) → bool

prosuite.utils.to_bool(value: Optional[Union[str, bool, int]] = None, default_value: bool = False) → bool

” treats “true” or “True” or ‘yes’ or ‘Yes’ or 1 or ‘1’ as True (bool). Anything else returns False (bool)

prosuite.utils.to_float(value: Optional[Union[str, int, float]], default_value: float = 0) → Optional[float]

Parameters

• value – value that gets converted to float type

• default_value – value that is returned in case the float conversion fails

Returns

prosuite.utils.to_int(value: Optional[Union[str, int]], default_value: int = 0) → Optional[int]

Tries to convert the input value to int. If conversion is not possible, returns the default value.

Parameters

• value – value that gets converted to int type

• default_value – value that is returned in case the int conversion fails

Returns

int representation of the input value or 0

prosuite.utils.to_spatial_reference_xml(sr_id: str)

prosuite.utils.try_get_from_oe(prop: str, object: lxml.objectify.ObjectifiedElement) → Optional[str]

prosuite.utils.try_get_from_str_dict(key: str, dictionary: dict, default_value: Optional[Union[int, float, str]] = '') → Optional[Union[int, float, str]]

returns the value of the lookup element if it is available in the dict. if the lookup element is not in the dict, the default value is returned.

prosuite.utils.try_get_lxml_attrib(node: lxml.objectify.ObjectifiedElement, attribute_name: str) → Optional[str]

If the attribute exists on the objectified node then return it, else return None

Organisational modules - Subpackages