List functions ​

List functions work on ordered collections. Most integration work is list work. A webhook delivers an array of records, an API returns pages of results, and a destination system expects a different shape, a different order, or a single aggregated number.

_ is the input your expression receives in these examples. Refer to Input variables for more information.

Call a list function ​

Call a function directly, or pipe a value into it with >>:

count(_.line_items)
_.line_items >> count

List functions chain well, and the pipe form is what makes a multi-step transformation readable:

_.orders >> filter_by(o ~> o.status == 'open') >> map_by(o ~> o.id)

Lambdas ​

Many list functions take a lambda, a small inline function written with ~>:

_.orders >> filter_by(order ~> order.total > 100)

Choose any name for the parameter before ~>. It refers to one element at a time. Wrap two or more parameters in parentheses (()):

_.items >> map_with_index_by((item, index) ~> f"{index + 1}. {item.name}")

Inspect a list ​

The following functions read a list without changing it:

count ​

Returns the number of elements.

count(list)

count(['SO-1', 'SO-2', 'SO-3'])

3

count([])

0

first ​

Returns the first element, or null if the list is empty.

first(list)

first(['pending', 'shipped'])

pending

first([])

null

last ​

Returns the last element, or null if the list is empty.

last(list)

last(['pending', 'shipped'])

shipped

at ​

Returns the element at a zero-based index. A negative index counts back from the end.

An index outside the list raises E201. Use first or last for just an end. Use find_by to search for an element rather than a position.

at(list, index)

at(['pending', 'shipped', 'closed'], 0)

pending

at(['pending', 'shipped', 'closed'], -1)

closed

one! ​

Returns the single element of a one-element list, and raises E204 for any other length.

Use it as an assertion. one! turns "zero or many" into a job that fails loudly instead of a transformation that quietly proceeds with the wrong record when a lookup must match exactly one record.

one!(list)

one!(['SO-1001'])

SO-1001

Test a list ​

The following functions answer a question about a list rather than returning a new one:

all? ​

Returns true if every element satisfies the predicate. An empty list returns true.

all?(list, predicate)

['open', 'open'] >> all?(s ~> s == 'open')

true

any? ​

Returns true if at least one element satisfies the predicate.

any?(list, predicate)

[199, 42, 8] >> any?(n ~> n > 100)

true

find_by ​

Returns the first element satisfying the predicate, or null if none does.

find_by(list, predicate)

[{id: 1, sku: 'WID-1'}, {id: 2, sku: 'GAD-7'}] >> find_by(r ~> r.sku == 'GAD-7')

{id: 2, sku: "GAD-7"}

find_first_index ​

Returns the zero-based index of the first element satisfying the predicate, or -1 if none does.

find_first_index(list, predicate)

['a', 'b', 'c'] >> find_first_index(s ~> s == 'c')

2

['a', 'b'] >> find_first_index(s ~> s == 'z')

-1

Select elements ​

The following functions return a shorter list built from the original:

filter_by ​

Keeps the elements for which the predicate returns true.

Also accepts a Map, in which case the lambda receives two parameters, the key and the value. The result is a Map too.

filter_by(list, predicate)

[1, 2, 3, 4] >> filter_by(n ~> n > 2)

[3, 4]

take ​

Returns the first count elements, or the last count elements if count is negative. A request for more than the list holds returns the whole list rather than raising an error.

take(list, count)

take(['a', 'b', 'c', 'd'], 2)

["a", "b"]

take(['a', 'b', 'c', 'd'], -2)

["c", "d"]

drop ​

Removes the first count elements, or the last count elements if count is negative.

drop(list, count)

drop(['a', 'b', 'c', 'd'], 2)

["c", "d"]

compact ​

Removes null and Skip values.

Useful directly after a map_by that produces a value only for some elements, and for cleaning optional fields out of an inbound payload.

compact(list)

compact(['a', null, 'b'])

["a", "b"]

unique ​

Removes duplicates, keeping the first occurrence of each value in its original position.

unique(list)

unique(['WID-1', 'GAD-7', 'WID-1'])

["WID-1", "GAD-7"]

unique_by ​

Removes duplicates by a key function, keeping the first element for each key.

Use it with records, where two entries are "the same" because they share an identifier even though the whole objects differ.

unique_by(list, key)

[{sku: 'WID-1'}, {sku: 'WID-1'}] >> unique_by(r ~> r.sku)

[{sku: "WID-1"}]

Use case: Keep the valid records from a webhook batch ​

A webhook delivers a batch in which some records are missing an email address and one identifier repeats. Use list functions to keep one record per identifier and drop the ones with no email to deliver to:

Input

{
  "contacts": [
    {"id": "C-1", "email": "[email protected]"},
    {"id": "C-2", "email": null},
    {"id": "C-1", "email": "[email protected]"},
    {"id": "C-3", "email": "[email protected]"}
  ]
}

Formula

_.contacts
  >> unique_by(c ~> c.id)
  >> filter_by(c ~> present?(c.email))
  >> map_by(c ~> c.email)

Output

["[email protected]", "[email protected]"]

unique_by runs before filter_by so the duplicate is removed on identity, not on whether it happened to have an email address.

Transform elements ​

The following functions return a list of the same length, or reshape the structure:

map_by ​

Applies a lambda to every element and returns the results.

An element whose lambda returns Skip is omitted, so map_by can filter and transform in the same pass.

map_by(list, transform)

[1, 2, 3] >> map_by(n ~> n * 2)

[2, 4, 6]

map_with_index_by ​

Applies a two-parameter lambda receiving the element and its zero-based index.

map_with_index_by(list, transform)

['a', 'b'] >> map_with_index_by((s, i) ~> f"{i + 1}. {s}")

["1. a", "2. b"]

flatten ​

Flattens one level of nesting, and only one level. A list nested two deep keeps its inner list, so call flatten again to go further.

flatten(list)

flatten([[1, 2], [3]])

[1, 2, 3]

[[1, 2], [3, [4]]] >> flatten

[1, 2, 3, [4]]

zip ​

Pairs elements from two lists into two-element lists. The shorter list determines the length.

zip(first, second)

zip(['sku', 'qty'], ['WID-1', 3])

[["sku", "WID-1"], ["qty", 3]]

zip(['a', 'b', 'c'], [1])

[["a", 1]]

reverse ​

Reverses the order of elements.

reverse(list)

reverse(['a', 'b', 'c'])

["c", "b", "a"]

Order a list ​

The following functions sort a list, ascending and stable:

sort ​

Sorts in ascending order. The sort is stable, so elements that compare equal keep their original relative order.

sort(list)

sort([3, 1, 2])

[1, 2, 3]

[3, 1, 2] >> sort >> reverse

[3, 2, 1]

sort_by ​

Sorts by a key function, ascending and stable.

There is no descending variant. Pipe the result into reverse for descending order.

sort_by(list, key)

[{n: 'b'}, {n: 'a'}] >> sort_by(r ~> r.n)

[{n: "a"}, {n: "b"}]

Use case: Take the three largest orders ​

A report needs the three largest orders pulled from a batch. Sort ascending, reverse, then take from the top:

Input

{
  "orders": [
    {"id": "SO-1", "amount": 120},
    {"id": "SO-2", "amount": 940},
    {"id": "SO-3", "amount": 310},
    {"id": "SO-4", "amount": 75}
  ]
}

Formula

_.orders
  >> sort_by(o ~> o.amount)
  >> reverse
  >> take(3)
  >> map_by(o ~> o.id)

Output

["SO-2", "SO-3", "SO-1"]

Group and split ​

The following functions divide a list into groups or two buckets, based on a key or a predicate:

group_by ​

Groups elements into a Map keyed by the result of the key function. Each value is the list of elements sharing that key.

group_by(list, key)

[{sku: 'WID-1', q: 1}, {sku: 'GAD-7', q: 2}, {sku: 'WID-1', q: 5}] >> group_by(r ~> r.sku)

{"WID-1": [{sku: "WID-1", q: 1}, {sku: "WID-1", q: 5}], "GAD-7": [{sku: "GAD-7", q: 2}]}

partition_by ​

Splits a list into exactly two lists: the elements matching the predicate, then the elements that don't.

The result is always a two-element list, so read it with at(0) and at(1).

partition_by(list, predicate)

[1, 2, 3, 4] >> partition_by(n ~> n > 2)

[[3, 4], [1, 2]]

[1, 2, 3, 4] >> partition_by(n ~> n > 2) >> at(0)

[3, 4]

Use case: Split a batch result into succeeded and failed ​

A bulk API returns one record per row with a per-row status. Use partition_by to split them, so the failures can be routed to a retry path while the successes continue:

Input

{
  "results": [
    {"id": "A", "status": "ok"},
    {"id": "B", "status": "error"},
    {"id": "C", "status": "ok"}
  ]
}

Formula

let parts = _.results >> partition_by(r ~> r.status == 'ok')
do {
  succeeded: parts >> at(0) >> map_by(r ~> r.id),
  failed: parts >> at(1) >> map_by(r ~> r.id)
}

Output

{"succeeded": ["A", "C"], "failed": ["B"]}

let … do binds the split once so the list is partitioned a single time rather than twice.

Aggregate ​

The following functions reduce a list to one value:

sum ​

Adds the numeric elements. An empty list sums to 0.

sum(list)

sum([12.50, 99.00])

111.5

sum([Decimal('12.50'), Decimal('99.00')])

111.50

sum([])

0

avg ​

Returns the mean of the numeric elements. An empty list raises E207, because there is no meaningful average of nothing.

avg(list)

avg([10, 20, 30])

20

min ​

Returns the smallest element. Works on strings as well as numbers.

min(list)

min([5, 2, 9])

2

max ​

Returns the largest element. Works on strings as well as numbers.

max(list)

max([5, 2, 9])

9

max(['apple', 'pear'])

pear

reduce_by ​

Folds a list into a single value using an accumulator.

Use it when the result isn't a plain total, such as building a Map, concatenating, or carrying state across elements. Use sum and avg instead for a straight sum or average, because their names say what they mean.

reduce_by(list, initial, accumulate)

[1, 2, 3] >> reduce_by(0, (acc, n) ~> acc + n)

6

Use case: Total an order by SKU ​

Line items arrive one row per shipment, so the same SKU appears more than once. Use list functions to group them and total the quantity for each:

Input

{
  "line_items": [
    {"sku": "WID-1", "qty": 3},
    {"sku": "GAD-7", "qty": 1},
    {"sku": "WID-1", "qty": 2}
  ]
}

Formula

_.line_items
  >> group_by(li ~> li.sku)
  >> entries
  >> map_by(e ~> {key: e.key, value: e.value >> map_by(r ~> r.qty) >> sum})
  >> from_entries

Output

{"WID-1": 5, "GAD-7": 1}

group_by produces a Map of lists. entries turns it into a list of {key, value} pairs so each group can be summed. from_entries then puts it back together. Refer to Map functions for both.

Generate a list ​

The following functions produce a new list rather than transforming an existing one:

range ​

Builds a list of integers from start to end, inclusive. The list descends when start is greater than end.

range(start, end)

range(1, 5)

[1, 2, 3, 4, 5]

range(5, 1)

[5, 4, 3, 2, 1]

random_choice ​

Returns one randomly selected element.

The result differs on every evaluation, so don't use it for anything a later step must reproduce. Refer to Random functions for other ways to generate a value that doesn't need to repeat.

random_choice(list)

Related ​

• Map functions: Work with the Map that group_by returns.
• Deep map functions: Reach through deeply nested structures.
• Operators: More information about the pipeline family of operators.
• Error codes: Troubleshoot job failures such as E201, E204, and E207.