Here’s another useful function I keep around:
Everyone knows what map does, and what concat does. And what mapcat does.
The function definition for pmapcat above, does what mapcat does, except that by using pmap underneath, it does so in parallel. The semantics are a bit different:
First off, the first parameter is called batches (and not, say, coll, for collection). This means that instead of passing in a simple collection of items, you have to pass in a collection of collections, where each is a batch of items.
Correspondingly, the parameter f is the function that will be applied not to each item, but to each batch of items.
Usage of this might look something like this:
One thing to remember is that pmap uses the Clojure send-off pool to do it’s thing, so the usual caveats will apply wrt to how f should behave.
i’d use [f & batches] so you don’t have to pass a collection of collections. just pass however many collections you want after the function.
Usually, you’ll have a collection of items, which you’ll split up into batches using something like partition-all. So the most common use-case is to have a collection of collections… thus the signature.
mapcat takes [f & colls], so i’d just try to be consistent when creating similar functionality in pmapcat.
Fair enough.
Although, I almost always use (apply mapcat …) when I use mapcat… so this saves me another word
Still, consistency is nice.
Reblogged this on s-expressions and commented:
Cross-posted from Zolo Labs