Introduction
This example shows a very simple flow element which takes a date of birth and returns an star sign. Although a basic (and not all that useful) flow element, the example demonstrates how you can start to implement your own flow elements.
Download Example
The source code used in this example is available here:
Dependencies
The flow element will need a reference to the Pipeline core package
Select a tab to view language specific dependencies.
Create a new console project (.NET Core or Framework) and add a reference to the
FiftyOne.Pipeline.Core NuGet package.
The only dependency needed for Java is the
pipeline.core Maven package from com.51degrees. To include this, add the following to the <dependencies> section of the project's pom.xml: <dependency>
<groupId>com.51degrees</groupId>
<artifactId>pipeline.core</artifactId>
<version>4.2.0</version>
</dependency>
Add a dependency to the
fiftyone.pipeline.core composer package to the composer.json, and required it in the source with include("../vendor/autoload.php"); Add a dependency to the
fiftyone.pipeline.core NPM package to the package.json, and required it in the source with require("fiftyone.pipeline.core"). Data
The element data being added to the flow data by this flow element is a star sign. So this should have its own 'getter' in its element data.
Select a tab to view language specific element data implementation.
.NET supports interfaces, so the element data should have a public interface and an internal concrete implementation with easy 'setters'.
In this example, only one value is populated. So an interface IStarSign will extend IElementData to add a 'getter' for it.
namespace Examples.CustomFlowElement.Data
{
public interface IStarSignData : IElementData
{
string StarSign { get; }
}
}
Now the internal implementation of it will implement this 'getter' and add a 'setter' for the flow element to use.
internal class StarSignData : ElementDataBase, IStarSignData
{
public StarSignData(
ILogger<ElementDataBase> logger,
IPipeline pipeline)
: base(logger, pipeline)
{
}
public string StarSign
{
// Get the star sign from the internal IDictionary as a string.
get { return GetAs<string>("starsign"); }
// Add the star sign to the internal IDictionary.
set { base["starsign"] = value; }
}
}
Java supports interfaces, so the element data should have a public interface and a package private concrete implementation with easy 'setters'.
In this example, only one value is populated. So an interface StarSignData will extend ElementData to add a 'getter' for it.
public interface StarSignData extends ElementData {
String getStarSign();
}
Now the internal implementation of it will implement this 'getter' and add a 'setter' for the flow element to use.
class StarSignDataInternal extends ElementDataBase implements StarSignData {
public StarSignDataInternal(Logger logger, FlowData flowData) {
super(logger, flowData);
}
@Override
public String getStarSign() {
return getAs("starsign", String.class);
}
void setStarSign(String starSign) {
put("starsign", starSign);
}
}
Note that this concrete implementation of StarSignData sits in the same package as the flow element, not the StarSignData interface, as it only needs to be accessible by the flow element.
PHP's implementation of element data does not require concrete getters for IDE autocompletion, so
elementDataDictionary can be used. Node's implementation of element data does not require concrete getters for IDE autocompletion, so
elementDataDictionary can be used. Flow Element
Now the actual flow element needs to be implemented. For this, the flow element's base class can be used which deals with most of the logic. The element properties, evidence keys and the processing are all that need implementing.
Select a tab to view language specific flow element implementation.
First let's define a class which extends
FlowElementBase (and partially implements IFlowElement). This has the type arguments of IStarSignData - the interface extending element data which will be added to the flow data, and IElementPropertyMetaData - as we only need the standard metadata for element properties.
This needs a constructor matching the FlowElementBase class. So it takes a logger, and an element data factory which will be used to construct an IStarSignData:
public class SimpleFlowElement : FlowElementBase<IStarSignData, IElementPropertyMetaData>
{
public SimpleFlowElement(
ILogger<FlowElementBase<IStarSignData, IElementPropertyMetaData>> logger,
Func<IPipeline, FlowElementBase<IStarSignData, IElementPropertyMetaData>, IStarSignData> elementDataFactory)
: base(logger, elementDataFactory)
{
Init();
}
The Init method in this example will simply initialize a list of star signs with the start and end dates of each star sign and add each to a list of a new class named StarSign which has the following simple implementation:
internal class StarSign
{
public StarSign(string name, DateTime start, DateTime end)
{
Name = name;
Start = start.AddYears(-start.Year + 1);
End = end.AddYears(-end.Year + 1);
}
public string Name { get; private set; }
public DateTime Start { get; private set; }
public DateTime End { get; private set; }
}
Note that the year of the start and end date are both set to 1, as the year should be ignored, but the year 0 cannot be used in a DateTime.
The new Init method looks like this:
private void Init()
{
var starSigns = new List<StarSign>();
foreach (var starSign in _starSignData)
{
starSigns.Add(new StarSign(
starSign[0],
DateTime.ParseExact(starSign[1], @"yy/MM/dd", CultureInfo.InvariantCulture),
DateTime.ParseExact(starSign[2], @"yy/MM/dd", CultureInfo.InvariantCulture)));
}
_starSigns = starSigns;
}
Now the abstract methods can be implemented to create a functional flow element.
public class SimpleFlowElement : FlowElementBase<IStarSignData, IElementPropertyMetaData>
{
public SimpleFlowElement(
ILogger<FlowElementBase<IStarSignData, IElementPropertyMetaData>> logger,
Func<IPipeline, FlowElementBase<IStarSignData, IElementPropertyMetaData>, IStarSignData> elementDataFactory)
: base(logger, elementDataFactory)
{
Init();
}
private IList<StarSign> _starSigns;
private static string[][] _starSignData = {
new string[3] {"Aries", "01/03/21", "01/04/19"},
new string[3] {"Taurus","01/04/20","01/05/20"},
new string[3] {"Gemini","01/05/21","01/06/20"},
new string[3] {"Cancer","01/06/21","01/07/22"},
new string[3] {"Leo","01/07/23","01/08/22"},
new string[3] {"Virgo","01/08/23","01/09/22"},
new string[3] {"Libra","01/09/23","01/10/22"},
new string[3] {"Scorpio","01/10/23","01/11/21"},
new string[3] {"Sagittarius","01/11/22","01/12/21"},
new string[3] {"Capricorn","01/12/22","01/01/19"},
new string[3] {"Aquarius","01/01/20","01/02/18"},
new string[3] {"Pisces","01/02/19","01/03/20"}
};
private void Init()
{
var starSigns = new List<StarSign>();
foreach (var starSign in _starSignData)
{
starSigns.Add(new StarSign(
starSign[0],
DateTime.ParseExact(starSign[1], @"yy/MM/dd", CultureInfo.InvariantCulture),
DateTime.ParseExact(starSign[2], @"yy/MM/dd", CultureInfo.InvariantCulture)));
}
_starSigns = starSigns;
}
// The IStarSignData will be stored with the key "starsign" in the FlowData.
public override string ElementDataKey => "starsign";
// The only item of evidence needed is "date-of-birth".
public override IEvidenceKeyFilter EvidenceKeyFilter =>
new EvidenceKeyFilterWhitelist(new List<string>() { "date-of-birth" });
public override IList<IElementPropertyMetaData> Properties =>
new List<IElementPropertyMetaData>()
{
// The only property which will be returned is "starsign" which will be
// a string.
new ElementPropertyMetaData(this, "starsign", typeof(string), true)
};
protected override void ProcessInternal(IFlowData data)
{
// Create a new IStarSignData, and cast to StarSignData so the 'setter' is available.
StarSignData starSignData = (StarSignData)data.GetOrAdd(ElementDataKey, CreateElementData);
if (data.TryGetEvidence("date-of-birth", out DateTime dateOfBirth))
{
// "date-of-birth" is there, so set the star sign.
var monthAndDay = new DateTime(1, dateOfBirth.Month, dateOfBirth.Day);
foreach (var starSign in _starSigns)
{
if (monthAndDay > starSign.Start &&
monthAndDay < starSign.End)
{
// The star sign has been found, so set it in the
// results.
starSignData.StarSign = starSign.Name;
break;
}
}
}
else
{
// "date-of-birth" is not there, so set the star sign to unknown.
starSignData.StarSign = "Unknown";
}
}
protected override void ManagedResourcesCleanup()
{
// Nothing to clean up here.
}
protected override void UnmanagedResourcesCleanup()
{
// Nothing to clean up here.
}
}
}
First let's define a class which extends FlowElementBase (which partially implements FlowElement). This has the type arguments of StarSignData - the interface extending element data which will be added to the flow data, and ElementPropertyMetaData - as we only need the standard metadata for element properties.
This needs a constructor matching the FlowElementBase class. So it takes a logger, and an element data factory which will be used to construct a StarSignData:
public class SimpleFlowElement extends FlowElementBase<StarSignData, ElementPropertyMetaData> {
public SimpleFlowElement(
Logger logger,
ElementDataFactory<StarSignData> elementDataFactory) {
super(logger, elementDataFactory);
init();
}
The init method in this example will simply initialize a list of star signs with the start and end dates of each star sign and add each to a list of a new class named StarSign which has the following simple implementation:
public class StarSign {
private final Calendar end;
private final Calendar start;
private final String name;
public StarSign(String name, String start, String end) {
this.name = name;
this.start = Calendar.getInstance();
String[] startDate = start.split("/");
this.start.set(
0,
Integer.parseInt(startDate[1]) - 1,
Integer.parseInt(startDate[0]));
this.end = Calendar.getInstance();
String[] endDate = end.split("/");
this.end.set(
0,
Integer.parseInt(endDate[1]) - 1,
Integer.parseInt(endDate[0]));
}
public String getName() {
return name;
}
public Calendar getStart() {
return start;
}
public Calendar getEnd() {
return end;
}
}
Note that the year of the start and end date are both set to 0, as the year should be ignored.
The new init method looks like this:
private void init() {
List<StarSign> starSigns = new ArrayList<>();
for (String[] starSign : starSignData) {
starSigns.add(new StarSign(
starSign[0],
starSign[1],
starSign[2]));
}
this.starSigns = starSigns;
}
Now the abstract methods can be implemented to create a functional flow element.
public class SimpleFlowElement extends FlowElementBase<StarSignData, ElementPropertyMetaData> {
public SimpleFlowElement(
Logger logger,
ElementDataFactory<StarSignData> elementDataFactory) {
super(logger, elementDataFactory);
init();
}
private static final String[][] starSignData = {
{"Aries","21/03","19/04"},
{"Taurus","20/04","20/05"},
{"Gemini","21/05","20/06"},
{"Cancer","21/06","22/07"},
{"Leo","23/07","22/08"},
{"Virgo","23/08","22/09"},
{"Libra","23/09","22/10"},
{"Scorpio","23/10","21/11"},
{"Sagittarius","22/11","21/12"},
{"Capricorn","22/12","19/01"},
{"Aquarius","20/01","18/02"},
{"Pisces","19/02","20/03"}
};
private List<StarSign> starSigns;
private void init() {
List<StarSign> starSigns = new ArrayList<>();
for (String[] starSign : starSignData) {
starSigns.add(new StarSign(
starSign[0],
starSign[1],
starSign[2]));
}
this.starSigns = starSigns;
}
@Override
protected void processInternal(FlowData data) throws Exception {
// Create a new StarSignData, and cast to StarSignDataInternal so the 'setter' is available.
StarSignDataInternal elementData = (StarSignDataInternal)data.getOrAdd(getElementDataKey(),getDataFactory());
TryGetResult<Date> date = data.tryGetEvidence("date-of-birth", Date.class);
if (date.hasValue()) {
// "date-of-birth" is there, so set the star sign.
Calendar dob = Calendar.getInstance();
dob.setTime(date.getValue());
Calendar monthAndDay = Calendar.getInstance();
monthAndDay.set(
0,
dob.get(Calendar.MONTH),
dob.get(Calendar.DATE));
for (StarSign starSign : starSigns) {
if (monthAndDay.compareTo(starSign.getStart()) >= 0 &&
monthAndDay.compareTo(starSign.getEnd()) <= 0) {
elementData.setStarSign(starSign.getName());
break;
}
}
}
else
{
// "date-of-birth" is not there, so set the star sign to unknown.
elementData.setStarSign("Unknown");
}
}
@Override
public String getElementDataKey() {
// The StarSignData will be stored with the key "starsign" in the FlowData.
return "starsign";
}
@Override
public EvidenceKeyFilter getEvidenceKeyFilter() {
// The only item of evidence needed is "date-of-birth".
return new EvidenceKeyFilterWhitelist(
Collections.singletonList("date-of-birth"),
String.CASE_INSENSITIVE_ORDER);
}
@Override
public List<ElementPropertyMetaData> getProperties() {
// The only property which will be returned is "starsign" which will be
// an String.
return Collections.singletonList(
(ElementPropertyMetaData) new ElementPropertyMetaDataDefault(
"starsign",
this,
"starsign",
String.class,
true));
}
@Override
protected void managedResourcesCleanup() {
// Nothing to clean up here.
}
@Override
protected void unmanagedResourcesCleanup() {
// Nothing to clean up here.
}
}
First let's define a class which extends
flowElement:
class AstrologyFlowElement extends FlowElement
{
Now the abstract methods can be implemented to create a functional flow element.
class AstrologyFlowElement extends FlowElement
{
// datakey used to categorise data coming back from this
// FlowElement in a Pipeline
public $dataKey = "astrology";
// The processInternal function is the core working of a FlowElement.
// It takes FlowData, reads evidence and returns data.
public function processInternal($FlowData)
{
$result = [];
// Get the date of birth from the query string (submitted through
// a form on the client side)
$dateOfBirth = $FlowData->evidence->get("query.dateOfBirth");
if ($dateOfBirth) {
$dateOfBirth = explode("-", $dateOfBirth);
$month = $dateOfBirth[1];
$day = $dateOfBirth[2];
$result["starSign"] = getStarSign($month, $day);
}
// Serve some JavaScript to the user that will be used to save
// a cookie with the user's latitude in it
$result["getLatitude"] = "navigator.geolocation.getCurrentPosition(function(position) {
document.cookie = \"latitude=\" + position.coords.latitude;
loadHemisphere();
});";
// Get the latitude from the above cookie
$latitude = $FlowData->evidence->get("cookie.latitude");
// Calculate the hemisphere
if ($latitude) {
$result["hemisphere"] = $latitude > 0 ? "Northern" : "Southern";
}
$data = new ElementDataDictionary($this, $result);
$FlowData->setElementData($data);
}
public $properties = array(
"starSign" => array(
"type" => "string",
"description" => "the user's starsign"
),
"hemisphere" => array(
"type" => "string",
"description" => "the user's hemisphere"
),
"getLatitude" => array(
"type" => "javascript",
"description" => "JavaScript used to get a user's latitude"
)
);
public function getEvidenceKeyFilter()
{
// A filter (in this case a basic list) stating which evidence
// the FlowElement is interested in
return new BasicListEvidenceKeyFilter(["cookie.latitude", "query.dateOfBirth"]);
}
}
First let's define a class which extends
flowElement.
This needs a constructor matching the flowElement class which initializes the element:
// Astrology flowElement
class Astrology extends FiftyOnePipelineCore.FlowElement {
constructor () {
super(...arguments);
// datakey used to categorise data coming back from
// this flowElement in a pipeline
this.dataKey = 'astrology';
// A filter (in this case a basic list) stating which evidence the
// flowElement is interested in, in this case a query string
this.evidenceKeyFilter = new FiftyOnePipelineCore
.BasicListEvidenceKeyFilter(['query.dateOfBirth']);
// The properties list includes extra information about
// the properties available from a flowElement
this.properties = {
starSign: {
type: 'string',
description: "the user's starsign"
}
};
}
Now the abstract methods can be implemented to create a functional flow element.
//! [constructor]
// Astrology flowElement
class Astrology extends FiftyOnePipelineCore.FlowElement {
constructor () {
super(...arguments);
// datakey used to categorise data coming back from
// this flowElement in a pipeline
this.dataKey = 'astrology';
// A filter (in this case a basic list) stating which evidence the
// flowElement is interested in, in this case a query string
this.evidenceKeyFilter = new FiftyOnePipelineCore
.BasicListEvidenceKeyFilter(['query.dateOfBirth']);
// The properties list includes extra information about
// the properties available from a flowElement
this.properties = {
starSign: {
type: 'string',
description: "the user's starsign"
}
};
}
//! [constructor]
// Internal processing function
processInternal (flowData) {
const result = {};
// Get the date of birth from the query string
// (submitted through a form on the client side)
let dateOfBirth = flowData.evidence.get('query.dateOfBirth');
if (dateOfBirth) {
dateOfBirth = dateOfBirth.split('-');
const month = parseInt(dateOfBirth[1]);
const day = parseInt(dateOfBirth[2]);
result.starSign = getStarSign(month, day);
}
// Save the data into an extension of the elementData class
// (in this case a simple dictionary subclass)
const data = new FiftyOnePipelineCore.ElementDataDictionary({
flowElement: this, contents: result
});
// Set this data on the flowElement
flowData.setElementData(data);
}
}
Builder
Now the flow element needs one final thing, an element builder to construct it. This only needs to provide the flow element with a logger and an element data factory - as this example has no extra configuration options.
Select a tab to view language specific element builder implementation.
The element builder has one important method, and that is
Build. This returns a new flow element which the element builder provides with a logger and an element data factory.
The element data factory is implemented in the element builder class to make use of the same logger factory.
public class SimpleFlowElementBuilder
{
private ILoggerFactory _loggerFactory;
public SimpleFlowElementBuilder(ILoggerFactory loggerFactory)
{
_loggerFactory = loggerFactory;
}
public SimpleFlowElement Build()
{
return new SimpleFlowElement(
_loggerFactory.CreateLogger<SimpleFlowElement>(),
CreateData);
}
// This is the element data factory, and is called in the Process method of the element.
public IStarSignData CreateData(IPipeline pipeline, FlowElementBase<IStarSignData,IElementPropertyMetaData> element)
{
return new StarSignData(_loggerFactory.CreateLogger<StarSignData>(), pipeline);
}
}
The element builder has one important method, and that is
build. This returns a new flow element which the element builder provides with a logger and an element data factory.
The element data factory is implemented in the element builder class to make use of the same logger factory.
public class SimpleFlowElementBuilder {
private final ILoggerFactory loggerFactory;
public SimpleFlowElementBuilder(ILoggerFactory loggerFactory) {
this.loggerFactory = loggerFactory;
}
public SimpleFlowElement build() {
return new SimpleFlowElement(
loggerFactory.getLogger(SimpleFlowElement.class.getName()),
new ElementDataFactory<StarSignData>() {
@Override
public StarSignData create(
FlowData flowData,
FlowElement<StarSignData, ?> flowElement) {
return new StarSignDataInternal(
loggerFactory.getLogger(StarSignDataInternal.class.getName()),
flowData);
}
});
}
}
The PHP implementation does not use separate builder classes. Instead the options are provided by optional constructor parameters.
The Node implementation does not use separate builder classes. Instead the options are provided by optional constructor parameters.
Usage
Select a tab to view language specific usage.
This new flow element can now be added to a Pipeline and used like:
// Construct the engine.
var starSignElement = new SimpleFlowElementBuilder(_loggerFactory)
.Build();
// Construct the pipeline with the example engine.
var pipeline = new PipelineBuilder(_loggerFactory)
.AddFlowElement(starSignElement)
.Build();
var dob = new DateTime(1992, 12, 18);
// Create a new flow data.
using (var flowData = pipeline.CreateFlowData())
{
// Add the evidence and process the data.
flowData
.AddEvidence("date-of-birth", dob)
.Process();
// Now get the result of the processing.
Console.WriteLine($"With a date of birth of " +
$"{dob.ToString("dd/MM/yyy")}" +
$", your star sign is " +
$"{flowData.Get<IStarSignData>().StarSign}.");
}
to give an output of:
With a date of birth of 18/12/1992, your star sign is Sagittarius.
This new flow element can now be added to a Pipeline and used like:
SimpleFlowElement starSignElement =
new SimpleFlowElementBuilder(loggerFactory)
.build();
Pipeline pipeline = new PipelineBuilder(loggerFactory)
.addFlowElement(starSignElement)
.build();
Calendar dob = Calendar.getInstance();
dob.set(1992, Calendar.DECEMBER, 18);
try (FlowData flowData = pipeline.createFlowData()) {
flowData
.addEvidence("date-of-birth", dob.getTime())
.process();
System.out.println("With a date of birth of " +
new SimpleDateFormat("yyyy/MM/dd").format(dob.getTime()) +
", your star sign is " +
flowData.getFromElement(starSignElement).getStarSign() +
".");
}
to give an output of:
With a date of birth of 18/12/1992, your star sign is Sagittarius.
This new flow element can now be added to a Pipeline and used like:
// Add some callback settings for the page to make a request with extra evidence from the client side, in this case the same url with an extra query string.
$javascriptBuilderSettings = array(
"host" => "localhost:3000",
"protocol" => "http",
"endpoint" => "/?json"
);
// Make the Pipeline and add the element we want to it
$Pipeline = (new PipelineBuilder(["javascriptBuilderSettings"=>$javascriptBuilderSettings]))->add(new AstrologyFlowElement())->build();
$FlowData = $Pipeline->createFlowData();
// Add any information from the request (headers, cookies and additional
// client side provided information)
$FlowData->evidence->setFromWebRequest();
// Process the FlowData
$FlowData->process();
// The client side JavaScript calls back to this page
if (isset($_GET["json"])) {
header("Content-Type: application/json; charset=UTF-8");
echo json_encode($FlowData->jsonbundler->json);
return;
}
// Generate the HTML for the form that gets a user's starsign
$output = "";
$output .= "<h1>Starsign</h1>";
$output .= "<form><label for='dateOfBirth'>Date of birth</label><input type='date' name='dateOfBirth' id='dateOfBirth'><input type='submit'></form>";
// Add the results if they're available
if ($FlowData->astrology->starSign) {
$output .= "<p>Your starsign is " . $FlowData->astrology->starSign . "</p>";
}
$output .= "<div id='hemispheretext'>";
if ($FlowData->astrology->hemisphere) {
$output .= "<p>Look at the " . $FlowData->astrology->hemisphere . " hemisphere stars tonight!</p>";
}
$output .= "</div>";
$output .= "<script>";
// This function will fire when the JSON data object is updated
// with information from the server.
// The sequence is:
// 1. Response contains JavaScript property 'getLatitude' that gets executed on the client
// 2. This triggers another call to the webserver that passes the location as evidence
// 3. The web server responds with new JSON data that contains the hemisphere based on the location.
// 4. The JavaScript integrates the new JSON data and fires the onChange callback below.
$output .= $FlowData->javascriptbuilder->javascript;
$output .= 'loadHemisphere = function() {
fod.complete(function (data) {
if(data.astrology.hemisphere) {
var para = document.createElement("p");
var text = document.createTextNode("Look at the " +
data.astrology.hemisphere + " hemisphere stars tonight");
para.appendChild(text);
var element = document.getElementById("hemispheretext");
var child = element.lastElementChild;
while (child) {
element.removeChild(child);
child = element.lastElementChild;
}
element.appendChild(para);
}
})};';
$output .= "</script>";
// Return the full output to the page
echo $output;
To print the star sign of the user.
This new flow element can now be added to a Pipeline and used like:
const element = new Astrology();
// Create the pipeline and add the astrology flowElement
const pipeline = new FiftyOnePipelineCore.PipelineBuilder()
.add(element)
.build();
// Create the flowData object used to set evidence on and get results from
const flowData = pipeline.createFlowData();
// Add the date of birth evidence
flowData.evidence.add('query.dateOfBirth', '2019-09-26');
// Process the flowData
flowData.process().then(function () {
// Get the starsign from the results
console.log(flowData.astrology.starSign);
});
To print the star sign of the user.
This new flow element can now be added to a Pipeline and used like:
SimpleFlowElement starSignElement =
new SimpleFlowElementBuilder(loggerFactory)
.build();
Pipeline pipeline = new PipelineBuilder(loggerFactory)
.addFlowElement(starSignElement)
.build();
Calendar dob = Calendar.getInstance();
dob.set(1992, Calendar.DECEMBER, 18);
try (FlowData flowData = pipeline.createFlowData()) {
flowData
.addEvidence("date-of-birth", dob.getTime())
.process();
System.out.println("With a date of birth of " +
new SimpleDateFormat("yyyy/MM/dd").format(dob.getTime()) +
", your star sign is " +
flowData.getFromElement(starSignElement).getStarSign() +
".");
}
to give an output of:
With a date of birth of 18/12/1992, your star sign is Sagittarius.
Next Steps
The example for Custom On-premise Engine shows how you can extend the functionality of this flow element to use a data file.
The flow element can also be upgraded to use evidence supplied by the client, as shown in the Custom Client-Side Evidence example.
To delegate the processing and logic to a cloud service, refer to the Custom Cloud Engine example.