Build your own

Add your own data service or custom engine to Pipeline API, or derive new properties from the ones it already returns with a short script. Combine the result with our device detection, IP intelligence, and geolocation data in one response.

An example element, a date of birth in and a star sign out

{
  "dateofbirth": "1992-11-05",
  "starsign": "Scorpio"
}

Example A custom flow element that turns a date of birth into a star sign

What you need to know

From evidence to one result

🔗

A pipeline is a chain of elements. The evidence from a request, such as its headers and IP address, goes in at one end. Each element adds its properties, and an element can read the properties the elements before it added.

Your application reads every property from one result, whichever element added it.

Read about Pipeline API

Your element in the pipeline

🔗

This example shows a very simple flow element which takes a date of birth and returns a star sign. Although a basic (and not all that useful) flow element, the example demonstrates how you can start to implement your own flow elements.

On-premise custom flow element, C#

public class SimpleFlowElement
    : FlowElementBase<IStarSignData, IElementPropertyMetaData>
{
    public override string ElementDataKey => "starsign";
    public override IEvidenceKeyFilter EvidenceKeyFilter =>
        new EvidenceKeyFilterWhitelist(
            new List<string>() { "date-of-birth" });
    protected override void ProcessInternal(IFlowData data)
    {
        StarSignData starSignData = (StarSignData)data.GetOrAdd(
            ElementDataKey, CreateElementData);
        if (data.TryGetEvidence(
            "date-of-birth", out DateTime dateOfBirth))
        {
            var monthAndDay = new DateTime(
                1, dateOfBirth.Month, dateOfBirth.Day);
            foreach (var starSign in _starSigns)
            {
                if (monthAndDay > starSign.Start &&
                    monthAndDay < starSign.End)
                {
                    starSignData.StarSign = starSign.Name;
                    break;
                }
            }
        }
        else
        {
            starSignData.StarSign = "Unknown";
        }
    }
}

The full C# on-premise example

On-premise custom flow element, Java

public class SimpleFlowElement
    extends FlowElementBase<StarSignData,
        ElementPropertyMetaData> {
    @Override
    protected void processInternal(FlowData data)
            throws Exception {
        StarSignDataInternal elementData =
            (StarSignDataInternal)data.getOrAdd(
                getElementDataKey(),getDataFactory());
        TryGetResult<Date> date =
            data.tryGetEvidence("date-of-birth", Date.class);
        if (date.hasValue()) {
            Calendar dob = Calendar.getInstance();
            dob.setTime(date.getValue());
            Calendar monthAndDay = Calendar.getInstance();
            monthAndDay.set(
                0,
                dob.get(Calendar.MONTH),
                dob.get(Calendar.DATE));
            for (StarSign starSign : starSigns) {
                if (monthAndDay.compareTo(
                        starSign.getStart()) >= 0 &&
                    monthAndDay.compareTo(
                        starSign.getEnd()) <= 0) {
                    elementData.setStarSign(starSign.getName());
                    break;
                }
            }
        }
        else
        {
            elementData.setStarSign("Unknown");
        }
    }
    @Override
    public String getElementDataKey() {
        return "starsign";
    }
    @Override
    public EvidenceKeyFilter getEvidenceKeyFilter() {
        return new EvidenceKeyFilterWhitelist(
            Collections.singletonList("date-of-birth"),
            String.CASE_INSENSITIVE_ORDER);
    }
}

The full Java on-premise example

On-premise custom flow element, Node.js

class Astrology extends FiftyOnePipelineCore.FlowElement {
  constructor () {
    super(...arguments);
    this.dataKey = 'astrology';
    this.evidenceKeyFilter = new FiftyOnePipelineCore
      .BasicListEvidenceKeyFilter(['query.dateOfBirth']);
  }
  processInternal (flowData) {
    const result = {};
    let dateOfBirth = flowData.evidence.get('query.dateOfBirth');
    if (dateOfBirth) {
      dateOfBirth = dateOfBirth.split('-');
      const month = parseInt(dateOfBirth[1]);
      const day = parseInt(dateOfBirth[2]);
      result.starSign = getStarSign(month, day);
    }
    const data = new FiftyOnePipelineCore.ElementDataDictionary({
      flowElement: this, contents: result
    });
    flowData.setElementData(data);
  }
}

The full Node.js on-premise example

On-premise custom flow element, Python

class AstrologyFlowElement(FlowElement):
    def __init__(self):
        super(AstrologyFlowElement, self).__init__()
        self.datakey = "astrology"
    def process_internal(self, flowdata):
        result = {}
        dateOfBirth = flowdata.evidence.get("query.dateOfBirth")
        if dateOfBirth:
            dateOfBirth = dateOfBirth.split('-')
            month = int(dateOfBirth[1])
            day = int(dateOfBirth[2])
            result["starSign"] = getStarSign(month, day)
        data = ElementDataDictionary(self, result)
        flowdata.set_element_data(data)
    def get_evidence_key_filter(self):
        return  BasicListEvidenceKeyFilter(
            ["cookie.latitude", "query.dateOfBirth"])

The full Python on-premise example

On-premise custom flow element, PHP

class AstrologyFlowElement extends FlowElement
{
    public string $dataKey = 'astrology';
    public function processInternal($flowData): void
    {
        $result = [];
        $dateOfBirth =
            $flowData->evidence->get('query.dateOfBirth');
        if ($dateOfBirth) {
            $dateOfBirth = explode('-', $dateOfBirth);
            $month = $dateOfBirth[1];
            $day = $dateOfBirth[2];
            $result['starSign'] = getStarSign($month, $day);
        }
        $data = new ElementDataDictionary($this, $result);
        $flowData->setElementData($data);
    }
    public function getEvidenceKeyFilter():
        BasicListEvidenceKeyFilter
    {
        return new BasicListEvidenceKeyFilter(
            ['cookie.latitude', 'query.dateOfBirth']);
    }
}

The full PHP on-premise example

On-premise custom flow element, Rust

const DATE_OF_BIRTH_EVIDENCE: &str = "date-of-birth";

impl FlowElement for SimpleStarSignElement {
    fn process(&self, data: &mut FlowData)
        -> Result<(), fiftyone_pipeline_core::Error> {
        let date = data
            .evidence()
            .get(DATE_OF_BIRTH_EVIDENCE)
            .map(str::to_owned);
        let sign = date
            .as_deref()
            .and_then(
                pipeline_examples::star_sign::parse_day_month)
            .and_then(|(month, day)|
                star_sign_for(&self.boundaries, month, day))
            .unwrap_or(UNKNOWN_STAR_SIGN)
            .to_owned();
        let star_sign_data =
            data.get_or_add(Self::KEY, StarSignData::new)?;
        star_sign_data.set_star_sign(sign);
        Ok(())
    }
    fn data_key(&self) -> &str {
        STAR_SIGN_DATA_KEY
    }
    fn evidence_key_filter(&self) -> &dyn EvidenceKeyFilter {
        &self.filter
    }
}

The full Rust on-premise example

On-premise custom flow element, Go

const EvidenceDateOfBirth = "query.date-of-birth"

type StarSignElement struct {
    *core.BaseElement
}

func NewStarSignElement() *StarSignElement {
    e := &StarSignElement{BaseElement: core.NewBaseElement(
        "starsign",
        core.NewWhitelistFilter(EvidenceDateOfBirth),
        nil)}
    return e
}

func (e *StarSignElement) Process(fd core.FlowData) error {
    data := fd.GetOrAdd(e.DataKey(),
        func(core.FlowData) core.ElementData {
            return core.NewBaseElementData() })
    raw, ok := fd.Evidence().Get(EvidenceDateOfBirth)
    if !ok {
        data.Set("starsign", "Unknown")
        return nil
    }
    dob, err := starsign.ParseDate(fmt.Sprintf("%v", raw))
    if err != nil {
        return err
    }
    data.Set("starsign", starsign.SignForDate(
        dob.Month(), dob.Day()))
    return nil
}

The full Go on-premise example

Read the custom element example

New properties from a short script

🔗

A derived property is worked out by rules from properties that other elements have already returned, such as banding a number or combining two answers into one. Each one is a single YAML or JSON file, so adding a property takes no code.

The script below says whether a browser is a recent release. Where a property it names cannot be read, it returns no value and a message naming that property, rather than a guess.

A derived property script, YAML

Format: 1
Name: BrowserCurrency
Version: 0.1.0
Output:
  Name: BrowserCurrency
  Description: Whether the browser making the request is a recent release.
  ValueType: string
  IsList: false
  IsMandatory: true
  Category: General
  Values:
    - Name: Current
      Description: Released in the last twelve months.
    - Name: Dated
      Description: Released more than twelve months ago.
Checks:
  Known:  { Property: device.BrowserReleaseYear, Gt: 0 }
  Recent: { Property: device.BrowserReleaseAge,  Lt: 12 }
Rules:
  - When: { Failed: Checks, Eq: 0 }
    Then: Current
  - Else: Dated

Add scripts to a .NET pipeline with the FiftyOne.Pipeline.DerivedProperty package, and try one in your browser with the script tester.

Read about derived properties

One call for every service

🔗

You can combine our device detection and geolocation results with your internal user profile data in the same pipeline.

A single API call can return the data from the services you combine. Your application reads the combined response.

Read about Pipeline API

Sell your data service

🔗

If you want to offer your data service through Pipeline API, talk to us about deployment and sales support.

Talk to us

Pick one to find out more

Each heading opens a short answer, with code where there is some, and a link to the documentation for more.

Read the custom element example

Why build your own?

Sell your data services to others

If you want to offer your data service through Pipeline API, talk to us about deployment and sales support.

Add properties without code

Derived properties turn the properties you already have into new ones, with rules in one short file.

Streamline the data services you use