51Degrees Pipeline for Python

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine Class Reference

Inheritance diagram for fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine:

[legend]

Collaboration diagram for fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine:

[legend]

Public Member Functions

 __init__ (self, settings={})
 Constructor for CloudRequestEngine.
 add_requested_properties (self, names)
 Add to the properties each request asks for, after the engine is built.
 check_credentials (self)
 Refuse a combination of resource key, licence key and requested properties that the cloud service would refuse on every request, so a deployment sees one configuration error when the engine is built rather than a failure on every request it serves.
 get_credentials (self)
 The credentials to send with a request, as the keys the cloud service reads them from.
 flow_element_properties (self)
 evidence_keys (self)
 get_evidence_keys (self)
 Internal function for getting evidence keys used by cloud engines @rtype: dict.
 get_evidence_key_filter (self)
 Instance of EvidenceKeyFilter based on the evidence keys fetched from the cloud service by the private getEvidenceKeys() method.
 get_engine_properties (self)
 Internal method to get properties for cloud engines from the cloud service.
 validate_response (self, requests.Response cloud_response, check_for_error_messages=True)
 Validate the JSON response from the cloud service.
 masked (self, text)
 Text from or about a cloud answer with the licence key in use shown only as its last four characters.
 make_cloud_request (self, type, url, content=None)
 @type url: string
 process_internal (self, flowdata)
 Processing function for the CloudRequestEngine Makes a request to the cloud service with the supplied resource key and evidence and returns a JSON object that is then parsed by cloud engines placed later in the pipeline.
 is_cacheable (self, data)
 A response whose JSON has a top-level member named fodid, in any case, holds a 51Did created for the one request that asked for it, so it is neither stored in the cache nor served from it.
 report_properties_not_covered (self, result)
 Compare the properties a request asked for with the ones its answer carries, and warn once about any it left out.
 get_content (self, flowData)
 Generate the Content to send in the POST request.
 sendable_evidence (self, evidence)
 The evidence the engine sends, being every value in the flow data, whether an element accepted it or not, apart from the web request view.
 add_query_data (self, query_data, all_evidence, evidence)
 Add query data to the evidence.
 get_selected_evidence (self, evidence, type)
 Get evidence with specified prefix.
 other_prefix_rank (self, key)
 The sort key of an evidence key that is not query, header or cookie evidence, where a lower one wins.
 key_has_prefix (self, itemKey, prefix)
 Check that the key of a KeyValuePair has the given prefix.

Public Member Functions inherited from fiftyone_pipeline_engines.engine.Engine

 __init__ (self, data_file=None)
 Constructor for an engine.
 set_cache (self, cache)
 Add a cache to an engine @type cashe: Cache.
 set_restricted_properties (self, properties_list)
 Add a subset of properties.
 in_cache (self, flowdata)
 A method to check if a flowdata's evidence is in the cache.
 refresh (self, identifier)
 process (self, flowdata)
 Engine's core process function.
 on_registration (self, pipeline)
 Called when an engine is registered with a pipeline and if there is a data file, a data file update service is attached to the parent pipeline.
 close (self)
 Stop the checks and updates of the engine's data file, whose timers would otherwise keep the engine alive.
 register_data_file (self, data_file)
 Register a data_file of the DataFile class with the engine @type data_file: DataFile.

Public Member Functions inherited from fiftyone_pipeline_core.flowelement.FlowElement

 __init__ (self)
 List of Pipelines the FlowElement has been added to.
 filter_evidence (self, flowdata)
 Filter FlowData evidence using the FlowElement's EvidenceKeyFilter.
 filter_evidence_key (self, key)
 Filter FlowData.evidence using the flowElement's EvidenceKeyFilter with the property key of evidence of interest.
 get_properties (self)
 Get the FlowElement.properties of a FlowElement.

Static Public Member Functions

 setting_text (settings, name)
 A setting as trimmed text, or an empty string when it is missing, None or only whitespace.
 setting_names (settings, name)
 A setting holding property names, as a list of trimmed, non-empty, lower-cased names.

Public Attributes

 resource_key = self.setting_text(settings, "resource_key")
str license_key = self.setting_text(settings, "license_key")
 requested_properties
 reported_not_covered = set()
str baseURL = settings["cloud_endpoint"]
 http_client = settings["http_client"]
 cloud_request_origin = settings["cloud_request_origin"]
 evidence_keys

Public Attributes inherited from fiftyone_pipeline_engines.engine.Engine

 data_file = data_file
 cache = cache
 restricted_properties = properties_list
dict restricted_property_keys

Public Attributes inherited from fiftyone_pipeline_core.flowelement.FlowElement

list pipelines = []
dict properties = {}
str datakey = ""
bool exclude_from_messages = False

Additional Inherited Members

Protected Member Functions inherited from fiftyone_pipeline_core.flowelement.FlowElement

 _log (self, level, message)

Constructor & Destructor Documentation

◆ __init__()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.__init__ ( self,
settings = {} )

Constructor for CloudRequestEngine.

@type settings: dict

Parameters
settingsSettings should contain a resource_key or a license_key, and optionally 1) requested_properties, the fully qualified names of the properties to ask for, as a list or a comma separated string. Required with a license_key alone. Beside a resource_key the cloud service adds them to the properties the resource key carries 2) a cloud_endpoint to overwrite the default baseurl 3) an cloud_request_origin to use when making requests

See the Credentials section of the cloud request engine specification for the combinations that are accepted: https://github.com/51Degrees/specifications/blob/main/pipeline-specification/pipeline-elements/cloud-request-engine.md#credentials

Member Function Documentation

◆ add_query_data()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.add_query_data ( self,
query_data,
all_evidence,
evidence )

Add query data to the evidence.

Parameters
query_data: The destination dictionary to add query data to.
all_evidenceAll evidence in the flow data. This is used to report which evidence keys are conflicting.
evidenceEvidence to add to the query Data.

◆ add_requested_properties()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.add_requested_properties ( self,
names )

Add to the properties each request asks for, after the engine is built.

The names are trimmed and lower-cased as the constructor does, so a name given in the case the documentation uses still matches the answer.

@type names: list or string

Parameters
namesthe names to add, as a list or one comma separated string, each as "section.property"

◆ check_credentials()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.check_credentials ( self)

Refuse a combination of resource key, licence key and requested properties that the cloud service would refuse on every request, so a deployment sees one configuration error when the engine is built rather than a failure on every request it serves.

◆ evidence_keys()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.evidence_keys ( self)

◆ flow_element_properties()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.flow_element_properties ( self)

◆ get_content()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.get_content ( self,
flowData )

Generate the Content to send in the POST request.

One name under several prefixes is sent once, with the value of the prefix that evidence.md ranks highest, being query, header, cookie, server, fiftyone and location, then any other prefix alphabetically. The groups are added lowest precedence first, so a later value replaces an earlier one, and a replacement by a key other than a query key is logged as a warning.

The engine sends every evidence value in the flow data, including the values no element in the pipeline accepts, because what the cloud service uses changes over time and knowing about the other evidence is important to continued optimization. The reserved names and the web request view are left out.

Parameters
flowData: FlowData
Returns
: Evidence Dictionary

◆ get_credentials()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.get_credentials ( self)

The credentials to send with a request, as the keys the cloud service reads them from.

A key that is not set is left out.

@rtype: dict

Returns
: Returns the credentials to send

◆ get_engine_properties()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.get_engine_properties ( self)

Internal method to get properties for cloud engines from the cloud service.

@rtype: dict

Returns
: Returns properties for all engines

◆ get_evidence_key_filter()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.get_evidence_key_filter ( self)

Instance of EvidenceKeyFilter based on the evidence keys fetched from the cloud service by the private getEvidenceKeys() method.

@type: BasicListEvidenceKeyFilter

Returns
: Returns BasicListEvidenceKeyFilter

Reimplemented from fiftyone_pipeline_core.flowelement.FlowElement.

◆ get_evidence_keys()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.get_evidence_keys ( self)

Internal function for getting evidence keys used by cloud engines @rtype: dict.

Returns
: Returns list of keys

◆ get_selected_evidence()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.get_selected_evidence ( self,
evidence,
type )

Get evidence with specified prefix.

Parameters
evidenceAll evidence in the flow data.
typeRequired evidence key prefix

◆ is_cacheable()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.is_cacheable ( self,
data )

A response whose JSON has a top-level member named fodid, in any case, holds a 51Did created for the one request that asked for it, so it is neither stored in the cache nor served from it.

A fodid below the top level is not one, and a response that cannot be read as a JSON object is cached as before.

@type data: ElementData

Parameters
dataA response the engine has just made, or one found in the cache

@rtype: bool

Returns
: False if the response holds a fodid

Reimplemented from fiftyone_pipeline_engines.engine.Engine.

◆ key_has_prefix()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.key_has_prefix ( self,
itemKey,
prefix )

Check that the key of a KeyValuePair has the given prefix.

Parameters
itemKeyKey to check
prefixThe prefix to check for.
Returns
: True if the key has the prefix.

◆ make_cloud_request()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.make_cloud_request ( self,
type,
url,
content = None )

@type url: string

Parameters
url

@rtype: dict

Returns
Returns dict with data and error properties error contains any errors from the request, data contains the response

◆ masked()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.masked ( self,
text )

Text from or about a cloud answer with the licence key in use shown only as its last four characters.

The cloud service names the key in some of its errors, and a URL may carry it in its query.

◆ other_prefix_rank()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.other_prefix_rank ( self,
key )

The sort key of an evidence key that is not query, header or cookie evidence, where a lower one wins.

server, fiftyone and location come first, in the order evidence.md lists them, then any other prefix alphabetically, all ignoring letter case.

Parameters
keyThe evidence key
Returns
: A tuple that sorts higher precedence first

◆ process_internal()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.process_internal ( self,
flowdata )

Processing function for the CloudRequestEngine Makes a request to the cloud service with the supplied resource key and evidence and returns a JSON object that is then parsed by cloud engines placed later in the pipeline.

@type FlowData: FlowData

Parameters
FlowDataReturns a JSON object that is then parsed by cloud engines

Reimplemented from fiftyone_pipeline_core.flowelement.FlowElement.

◆ report_properties_not_covered()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.report_properties_not_covered ( self,
result )

Compare the properties a request asked for with the ones its answer carries, and warn once about any it left out.

With a licence key, the cloud service drops a property the licence key does not cover without naming it, whenever the request also asked for a property the key does cover, so the engine has to notice the difference itself. With a resource key and no licence key, a property no licence grants comes back as null with a reason, so a name left out is not a property, and the warning says that instead. A property that is present but null counts as answered, because it carries a nullreason beside it saying why it has no value on this request, which is a different matter from entitlement. The answer cannot change while the keys and the list stay the same, so each distinct list is reported once.

@type result: string

Parameters
resultthe JSON text the cloud service answered with

◆ sendable_evidence()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.sendable_evidence ( self,
evidence )

The evidence the engine sends, being every value in the flow data, whether an element accepted it or not, apart from the web request view.

The reserved names are left out by add_query_data, which warns as it does so. An accepted value wins over a set aside one of the same key.

Parameters
evidenceThe flow data's Evidence
Returns
: The evidence keys and values to send

◆ setting_names()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.setting_names ( settings,
name )
static

A setting holding property names, as a list of trimmed, non-empty, lower-cased names.

The setting may be a list of names or one comma separated string, and a missing setting gives an empty list.

The names are lower-cased because the cloud service lower-cases property names in its answer, so "Device.IsMobile" comes back as "ismobile" under "device".

◆ setting_text()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.setting_text ( settings,
name )
static

A setting as trimmed text, or an empty string when it is missing, None or only whitespace.

◆ validate_response()

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.validate_response ( self,
requests.Response cloud_response,
check_for_error_messages = True )

Validate the JSON response from the cloud service.

@type: Response

Parameters
Response returned from the cloud service. @rtype: Exception
Returns
: Thrown if there are errors returned from the cloud service.

Member Data Documentation

◆ baseURL

str fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.baseURL = settings["cloud_endpoint"]

◆ cloud_request_origin

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.cloud_request_origin = settings["cloud_request_origin"]

◆ evidence_keys

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.evidence_keys

◆ http_client

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.http_client = settings["http_client"]

◆ license_key

str fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.license_key = self.setting_text(settings, "license_key")

◆ reported_not_covered

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.reported_not_covered = set()

◆ requested_properties

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.requested_properties
Initial value:
= self.setting_names(
settings, "requested_properties")

◆ resource_key

fiftyone_pipeline_cloudrequestengine.cloudrequestengine.CloudRequestEngine.resource_key = self.setting_text(settings, "resource_key")