Inconsistency between code&documentation of partition.

[fcr--]

Documentation says that partition returns {gen1, param1, state1}, {gen2, param2, state2}, yet what is really being returned is gen1, gen2, param2, state2.

[fcr--]

Probably the best bet is to update the documentation in order to avoid breaking code. Btw, split is already returning gen1, gen2, param2, state2 so I don't think maintaining the current partition behavior is a big issue.

[fcr--]

Btw, the documentation also mentions

Parameters: x – a value to find

yet the parameter is a predicate. So this was probably caused by an incorrect copy/paste.

[ligurio]

It seems so:

tarantool> function predicate(i, x) return i % 2 == 0 end                       
---     
...    
                                       
tarantool> predicate                                                            
---        
- 'function: 0x40d2f9b0'  
...                       
                                       
tarantool> range = fun.range(10)                                                
---
...

tarantool> range
---
- state: 0
  gen: 'function: 0x401ddf90'
  param:
  - 10
  - 1
...

tarantool> fun.partition(predicate, range)
---
- state: 0
  gen: 'function: 0x401e5ca0'
  param:
  - 'function: 0x40d2f9b0'
  - 'function: 0x401ddf90'
  - &0
    - 10
    - 1
- state: 0
  gen: 'function: 0x401e5ca0'
  param: &1
  - 'function: 0x418b6e68'
  - 'function: 0x401ddf90'
  - *0
- *1
- 0
...

tarantool> g1, g2, state, param = fun.partition(predicate, range)
---
...

tarantool> g1
---
- state: 0
  gen: 'function: 0x401e5ca0'
  param:
  - 'function: 0x40d2f9b0'
  - 'function: 0x401ddf90'
  - - 10
    - 1
...

tarantool> g2
---
- state: 0
  gen: 'function: 0x401e5ca0'
  param:
  - 'function: 0x407a15b8'
  - 'function: 0x401ddf90'
  - - 10
    - 1
...
tarantool> state
---
- - 'function: 0x407a15b8'
  - 'function: 0x401ddf90'
  - - 10
    - 1
...

tarantool> param
---
- 0
...

It is not critical, but may be confusing.

[ligurio]

The same for filter():

tarantool> g1, param, state = fun.filter(function(x) return x % 3 == 0 end, fun.range(10))
---
...

tarantool> g1
---
- state: 0
  gen: 'function: 0x41f93bc8'
  param:
  - 'function: 0x40551738'
  - 'function: 0x41f877f8'
  - - 10
    - 1
...

tarantool> param
---
- - 'function: 0x40551738'
  - 'function: 0x41f877f8'
  - - 10
    - 1
...

tarantool> state
---
- 0
...

[Totktonada]

I see, the change is mostly about replacing :returns: an iterator with :returns: ``{gen, param, state}, param, state`` . I don't find it perfectly correct and I doubt that it makes the documentation more expressive. I'll explain.

Definitions

First, let's define <lua iterator>: it is the gen, param, state triplet¹. What does 'a function returns an iterator' means so? It returns the triplet. We would write it as return gen, param, state in code.

Next, let's define <luafun iterator> that wraps <lua iterator> (gen, param, state): it is the gen', param, state triplet, where gen' just calls gen when called, but also has luafun methods (like :take()).

Correctness

Okay, so whether :returns: an iterator is incorrect? Nope, all (or most of) luafun functions return the triplet, which is <luafun iterator> (and so it is <lua iterator> as well, the triplet). It seems valid to assume a luafun iterator behind the 'an iterator' words in context of the luafun documentation.

What would be incorrect? If we would say :returns: gen. The gen is one value, not a triplet.

I would also highlight that describing a luafun iterator as {gen, param, state}, param, state is not quite correct. The first item of the <luafun iterator> triplet is the {gen = <...>, param = <...>, state = <...>} mapping (with luafun's metatable), not the {gen, param, state} array (so you can't access gen using it[1]). Also I don't think that internal luafun iterator structure is so important to reveal it in a description of each method.

Actual mistakes

Sure, I see, there are places, where such {gen, param, state} description is already used. I guess that those places are actually traces of the old implementation (see 6c35d3c) and should be replaced with 'an iterator'.

See, there is also the following sentence in the partition documentation:

Iterators especially returned in tables to work with :func:zip and other functions.

It is the artifact of the old implementation as well.

Return two iterators (what is okay)

If we'll describe the partition return value as an iterator, an iterator (like it was done for span), whether it will be correct?

Let's consider the following snippet first:

tarantool> function x() return unpack({1, 2, 3}), unpack({'a', 'b', 'c'}) end
tarantool> x()
---
- 1
- a
- b
- c
...

Here we return two triplets and, according to the Lua language rules, only the last one gives more than one value. Our iterators are triplets too!

So the :returns: an iterator, an iterator description looks quite natural. A Lua programmer will expect it1, it2 if (s)he does not bother about triplets or it1, it2, param2, state2 if (s)he does.

Return two iterators (actual mistakes)

Not all persons who write on Lua so deep know it to keep all those multireturn details in the mind. Nicely, with luafun you usualy don't ever need to know about triplets and can consider the iterator as one value. Just use it:take(<...>):filter(<...>):totable() and so on.

When we return one luafun iterator from a function, it also works with the for <..> in <...> loop (because it is the triplet). However it does not work so nice, when we return two iterators:

tarantool> for i in fun.range(10):partition(function(i) return i % 2 == 0 end) do print(i) end
---
- error: 'builtin/fun.lua:544: attempt to call local ''gen_x'' (a nil value)'
...

I think that we should add a recipe for those functions (a note in each of them, at least partition and span):

Note: call :unwrap() on iterators to pass them into the for <..> in <...> loop. Example:

tarantool> it1, it2 = fun.range(10):partition(function(i) return i % 2 == 0 end)
tarantool> for i in it1:unwrap() do print(i) end
2
4
6
8
10
---
...

tarantool> for i in it2:unwrap() do print(i) end
1
3
5
7
9
---
...

To be precise, the function returns it1, it2, param2, state2, so the second iterator may be used in the for <...> in <...> loop without the :unwrap() call:

tarantool> it1, it2, param2, state2 = fun.range(10):partition(function(i) return i % 2 == 0 end)
tarantool> for i in it2, param2, state2 do print(i) end
1
3
5
7
9
---
...

(I use tarantool in examples, but it is better to adopt them to the luafun documentation style.)

Footnotes

1. Not a real definition (it lacks gen, param and state properties), but it is enough to explain what I want to explain. ↩