By memecs


2012-09-23 12:13:11 8 Comments

C++11 provides multiple ways to iterate over containers. For example:

Range-based loop

for(auto c : container) fun(c)

std::for_each

for_each(container.begin(),container.end(),fun)

However what is the recommended way to iterate over two (or more) containers of the same size to accomplish something like:

for(unsigned i = 0; i < containerA.size(); ++i) {
  containerA[i] = containerB[i];
}

9 comments

@Szymon Marczak 2017-07-28 11:43:29

I'm a bit late too; but you can use this (C-style variadic function):

template<typename T>
void foreach(std::function<void(T)> callback, int count, ...) {
    va_list args;
    va_start(args, count);

    for (int i = 0; i < count; i++) {
        std::vector<T> v = va_arg(args, std::vector<T>);
        std::for_each(v.begin(), v.end(), callback);
    }

    va_end(args);
}

foreach<int>([](const int &i) {
    // do something here
}, 6, vecA, vecB, vecC, vecD, vecE, vecF);

or this (using a function parameter pack):

template<typename Func, typename T>
void foreach(Func callback, std::vector<T> &v) {
    std::for_each(v.begin(), v.end(), callback);
}

template<typename Func, typename T, typename... Args>
void foreach(Func callback, std::vector<T> &v, Args... args) {
    std::for_each(v.begin(), v.end(), callback);
    return foreach(callback, args...);
}

foreach([](const int &i){
    // do something here
}, vecA, vecB, vecC, vecD, vecE, vecF);

or this (using a brace-enclosed initializer list):

template<typename Func, typename T>
void foreach(Func callback, std::initializer_list<std::vector<T>> list) {
    for (auto &vec : list) {
        std::for_each(vec.begin(), vec.end(), callback);
    }
}

foreach([](const int &i){
    // do something here
}, {vecA, vecB, vecC, vecD, vecE, vecF});

or you can join vectors like here: What is the best way to concatenate two vectors? and then iterate over big vector.

@Konrad Rudolph 2013-09-25 13:19:57

Rather late to the party. But: I would iterate over indices. But not with the classical for loop but instead with a range-based for loop over the indices:

for(unsigned i : indices(containerA)) {
    containerA[i] = containerB[i];
}

indices is a simple wrapper function which returns a (lazily evaluated) range for the indices. Since the implementation – though simple – is a bit too long to post it here, you can find an implementation on GitHub.

This code is as efficient as using a manual, classical for loop.

If this pattern occurs often in your data, consider using another pattern which zips two sequences and produces a range of tuples, corresponding to the paired elements:

for (auto& [a, b] : zip(containerA, containerB)) {
    a = b;
}

The implementation of zip is left as an exercise for the reader, but it follows easily from the implementation of indices.

(Before C++17 you’d have to write the following instead:)

for (auto items&& : zip(containerA, containerB))
    get<0>(items) = get<1>(items);

@SebastianK 2014-11-06 13:11:00

Is there any advantage of your indices implementation in comparison to boost counting_range? One could simply use boost::counting_range(size_t(0), containerA.size())

@Konrad Rudolph 2014-11-06 14:03:25

@SebastianK The biggest difference in this case is syntax: mine is (I claim) objectively better to use in this case. Furthermore, you can specify a step size. See the linked Github page, and in particular the README file, for examples.

@SebastianK 2014-11-06 15:11:08

Your idea is very nice and I came up with the use of counting_range only after seeing it: clear upvote :) However, I wonder if it provides additional value to (re-)implement this. E.g., regarding performance. Nicer syntax, I agree, of course, but it would be enough to write a simple generator function to compensate this drawback.

@Konrad Rudolph 2014-11-06 15:17:36

@SebastianK I admit that I when I wrote the code I deemed it simple enough to live in isolation without using a library (and it is!). Now I would probably write it as a wrapper around Boost.Range. That said, the performance of my library is already optimal. What I mean by this is that using my indices implementation yields compiler output which is identical to using manual for loops. There is no overhead whatsoever.

@SebastianK 2014-11-06 15:45:38

Since I use boost anyway, it would be simpler in my case. I already wrote this wrapper around boost range: a function with one line of code is all I need. However, I would be interested if the performance of boost ranges is optimal as well.

@Pixelchemist 2016-10-23 04:24:35

About your cpp11 range: Why do you have indices(std::initializer_list<T>&& cont)? Any rvalue should be "gone" by the time the range based for loop is entered.

@Konrad Rudolph 2016-10-24 09:58:51

@Pixelchemist To be fair the rvalue reference is completely unnecessary here (I’d use pass by value now). But it doesn’t hurt, either: the value of cont isn’t saved anywhere, and isn’t used inside the loop.

@cmaster - reinstate monica 2017-07-28 12:46:03

If you ask me, I find the line containerA[i] = containerB[i]; a lot clearer than the line get<0>(items) = get<1>(items). Why would I even want to do away with the explicit index? And, please, don't say "Performance with list containers!": If you care about performance, there won't be any std::list<> in your code anyway.

@Konrad Rudolph 2017-07-28 13:09:44

@cmaster It’s completely unrelated to performance but rather to robustness. Your preferred line has two potential off-by-one errors and potentially out-of-range index errors, which would cause UB. This should raise all the alarms. The line with get may not be as pretty but it’s, by contrast, completely safe from UB. It‘s preventing a common cause of hard-to-debug bugs.

@cmaster - reinstate monica 2017-07-28 20:42:12

@KonradRudolph ...at the cost of making the code harder to read, which, in turn, is also a major source of bugs. Yes, the zip() does defend against off-by-one errors. However, I prefer the better readability of the explicit index: In my experience, those off-by-one errors are extremely rare as writing correct loop control for just iterating over a container is an absolute no-brainer to me.

@Konrad Rudolph 2017-07-31 10:22:21

@cmaster I agree that readability is an important feature of code, and I’d prefer if the tuple-unpacking code were more readable. But it’s important to remember that it’s not a virtue in itself. Rather, readability just a means of making code more correct. Correctness is the ultimate goal, so in a trade-off between a (slight) increase in readability versus an increase in robustness, we should probably choose robustness.

@Konrad Rudolph 2017-07-31 10:23:46

@cmaster You say “In my experience, those off-by-one errors are extremely rare”. On the contrary: they are a major source of bugs. This is, after all, why so many modern features of programming languages (not just C++) are so eager to eliminate explicit loops and indexed access. It’s precisely to remove this extremely prevalent source of bugs.

@mathengineer 2018-11-23 10:20:57

indices only work if you include "range.hpp" at the github to add: template <typename C, typename = typename std::enable_if<traits::has_size<C>::value>> auto indices(C const& cont) -> range_proxy<decltype(cont.size())> { return {0, cont.size()}; }

@user877329 2017-04-18 07:54:28

Here is one variant

template<class ... Iterator>
void increment_dummy(Iterator ... i)
    {}

template<class Function,class ... Iterator>
void for_each_combined(size_t N,Function&& fun,Iterator... iter)
    {
    while(N!=0)
        {
        fun(*iter...);
        increment_dummy(++iter...);
        --N;
        }
    }

Example usage

void arrays_mix(size_t N,const float* x,const float* y,float* z)
    {
    for_each_combined(N,[](float x,float y,float& z){z=x+y;},x,y,z);    
    }

@Jens 2016-07-25 14:12:53

A range-library provides this and other very helpful functionality. The following example uses Boost.Range. Eric Niebler's rangev3 should be a good alternative.

#include <boost/range/combine.hpp>
#include <iostream>
#include <vector>
#include <list>

int main(int, const char*[])
{
    std::vector<int> const v{0,1,2,3,4};
    std::list<char> const  l{'a', 'b', 'c', 'd', 'e'};

    for(auto const& i: boost::combine(v, l))
    {
        int ti;
        char tc;
        boost::tie(ti,tc) = i;
        std::cout << '(' << ti << ',' << tc << ')' << '\n';
    }

    return 0;
}

C++17 will make this even better with structured bindings:

int main(int, const char*[])
{
    std::vector<int> const v{0,1,2,3,4};
    std::list<char> const  l{'a', 'b', 'c', 'd', 'e'};

    for(auto const& [ti, tc]: boost::combine(v, l))
    {
        std::cout << '(' << ti << ',' << tc << ')' << '\n';
    }

    return 0;
}

@syam 2016-11-14 21:26:13

This program is not compiling with g++ 4.8.0. delme.cxx:15:25: error: no match for 'operator=' (operand types are 'std::tuple<int&, char&>' and 'const boost::tuples::cons<const int&, boost::tuples::cons<const char&, boost::tuples::null_type> >') std::tie(ti,tc) = i; ^

@syam 2016-11-14 21:52:52

After changing std::tie to boost:tie, it compiled.

@pooya13 2019-03-16 21:36:32

I get the following compile error for the version with structured binding (using MSVC 19.13.26132.0 and Windows SDK version 10.0.16299.0): error C2679: binary '<<': no operator found which takes a right-hand operand of type 'const boost::tuples::cons<const char &,boost::fusion::detail::build_tuple_cons<boost::fusion::sin‌​gle_view_iterator<Se‌​quence,boost::mpl::i‌​nt_<1>>,Last,true>::‌​type>' (or there is no acceptable conversion)

@Dev Null 2019-04-09 06:08:38

structured bindings don't seem to work with boost::combine: stackoverflow.com/q/55585723/8414561

@Vahid 2016-06-09 20:15:04

another solution could be capturing a reference of the iterator of the other container in a lambda and using post increment operator on that. for example simple copy would be:

vector<double> a{1, 2, 3};
vector<double> b(3);

auto ita = a.begin();
for_each(b.begin(), b.end(), [&ita](auto &itb) { itb = *ita++; })

inside lambda you can do whatever with ita and then increment it. This easily extends to the multiple containers case.

@Joseph 2016-03-28 10:36:01

i wonder why no one mentioned this:

auto ItA = VectorA.begin();
auto ItB = VectorB.begin();

while(ItA != VectorA.end() || ItB != VectorB.end())
{
    if(ItA != VectorA.end())
    {
        ++ItA;
    }
    if(ItB != VectorB.end())
    {
        ++ItB;
    }
}

PS: if the container sizes don't match,then you will have to put the code inside the if statements.

@czarles 2014-04-29 19:01:23

In case when you need to iterate simultaneously over 2 containers only, there is an extended version of standard for_each algorithm in boost range library, e.g:

#include <vector>
#include <boost/assign/list_of.hpp>
#include <boost/bind.hpp>
#include <boost/range/algorithm_ext/for_each.hpp>

void foo(int a, int& b)
{
    b = a + 1;
}

int main()
{
    std::vector<int> contA = boost::assign::list_of(4)(3)(5)(2);
    std::vector<int> contB(contA.size(), 0);

    boost::for_each(contA, contB, boost::bind(&foo, _1, _2));
    // contB will be now 5,4,6,3
    //...
    return 0;
}

When you need to handle more than 2 containers in one algorithm, then you need to play with zip.

@Mikhail 2016-07-29 16:45:26

Wonderful! How did you find? Seems like it is not documented anywhere.

@Xeo 2012-09-23 15:04:59

For your specific example, just use

std::copy_n(contB.begin(), contA.size(), contA.begin())

For the more general case, you can use Boost.Iterator's zip_iterator, with a small function to make it usable in range-based for loops. For most cases, this will work:

template<class... Conts>
auto zip_range(Conts&... conts)
  -> decltype(boost::make_iterator_range(
  boost::make_zip_iterator(boost::make_tuple(conts.begin()...)),
  boost::make_zip_iterator(boost::make_tuple(conts.end()...))))
{
  return {boost::make_zip_iterator(boost::make_tuple(conts.begin()...)),
          boost::make_zip_iterator(boost::make_tuple(conts.end()...))};
}

// ...
for(auto&& t : zip_range(contA, contB))
  std::cout << t.get<0>() << " : " << t.get<1>() << "\n";

Live example.

However, for full-blown genericity, you probably want something more like this, which will work correctly for arrays and user-defined types that don't have member begin()/end() but do have begin/end functions in their namespace. Also, this will allow the user to specifically get const access through the zip_c... functions.

And if you're an advocate of nice error messages, like me, then you probably want this, which checks if any temporary containers were passed to any of the zip_... functions, and prints a nice error message if so.

@memecs 2012-09-23 16:12:17

Thanks! One question though, why do you use auto&&, what does it mean &&?

@Xeo 2012-09-23 16:18:43

@memecs: I recommend reading through this question, aswell as this answer of mine which kinda explains how the deduction and reference collapsing is done. Note that auto works exactly the same as a template parameter, and T&& in a template is a universal reference as explained in the first link, so auto&& v = 42 will be deduced as int&& and auto&& w = v; will then be deduced as int&. It allows you to match lvalues aswell as rvalues and let both be mutable, without making a copy.

@Viktor Sehr 2013-09-25 14:43:37

@Xeo: But whats the advantage of auto&& over auto& in a foreach loop?

@Xeo 2013-09-25 14:46:12

@ViktorSehr: It allows you to bind to temporary elements, like those produced by zip_range.

@Viktor Sehr 2013-09-25 15:07:55

@Xeo: I still don't get it, will I avoid a copy by using auto&& instead of auto& in this case?

@Xeo 2013-09-25 15:19:51

@ViktorSehr: auto& is an lvalue reference - it simply cannot bind to temporaries. auto&&, on the other hand, can bind to either temporaries or references (lvalues).

@Viktor Sehr 2013-09-26 08:23:42

@Xeo: Aha (I still cant figure why the compiler can't create a temprary like when you write const auto& value = some_func();

@Xeo 2013-09-26 09:00:06

@ViktorSehr: auto& != auto const&. The former only binds to lvalues.

@kynan 2015-11-23 12:00:58

@Xeo All the links to the examples are broken.

@wjl 2012-09-23 14:40:41

There are a lot of ways to do specific things with multiple containers as provided in the algorithm header. For instance, in the example you've given, you could use std::copy instead of an explicit for loop.

On the other hand, there isn't any built-in way to generically iterate multiple containers other than a normal for loop. This isn't surprising because there are a lot of ways to iterate. Think about it: you could iterate through one container with one step, one container with another step; or through one container until it gets to the end then start inserting while you go through to the end of the other container; or one step of the first container for every time you completely go through the other container then start over; or some other pattern; or more than two containers at a time; etc ...

However, if you wanted to make your own "for_each" style function that iterates through two containers only up to the length of the shortest one, you could do something like this:

template <typename Container1, typename Container2>
void custom_for_each(
  Container1 &c1,
  Container2 &c2,
  std::function<void(Container1::iterator &it1, Container2::iterator &it2)> f)
{
  Container1::iterator begin1 = c1.begin();
  Container2::iterator begin2 = c2.begin();
  Container1::iterator end1 = c1.end();
  Container2::iterator end2 = c2.end();
  Container1::iterator i1;
  Container1::iterator i2;
  for (i1 = begin1, i2 = begin2; (i1 != end1) && (i2 != end2); ++it1, ++i2) {
    f(i1, i2);
  }
}

Obviously you can make any kind of iterations strategy you want in a similar way.

Of course, you might argue that just doing the inner for loop directly is easier than writing a custom function like this ... and you'd be right, if you are only going to do it one or two times. But the nice thing is that this is very reusable. =)

@David Doria 2016-01-04 21:51:42

It seems like you have to declare the iterators before the loop? I tried this: for (Container1::iterator i1 = c1.begin(), Container2::iterator i2 = c2.begin(); (i1 != end1) && (i2 != end2); ++it1, ++i2) but the compiler yells. Can anyone explain why this is invalid?

@wjl 2016-01-04 22:24:47

@DavidDoria The first part of the for loop is a single statement. You can't declare two variables of different types in the same statement. Think about why for (int x = 0, y = 0; ... works, but for (int x = 0, double y = 0; ...) does not.

@lorro 2016-07-12 19:58:49

.. you can, however, have std::pair<Container1::iterator, Container2::iterator> its = {c1.begin(), c2.begin()};

@wjl 2016-07-18 02:06:42

Another thing to note is that this could be easily made variadic with C++14's typename...

Related Questions

Sponsored Content

5 Answered Questions

[SOLVED] How to iterate through two lists in parallel?

42 Answered Questions

[SOLVED] What's the best way to trim std::string?

  • 2008-10-19 19:23:07
  • Milan Babuškov
  • 658502 View
  • 782 Score
  • 42 Answer
  • Tags:   c++ trim stdstring

76 Answered Questions

[SOLVED] How do I iterate over the words of a string?

  • 2008-10-25 08:58:21
  • Ashwin Nanjappa
  • 2151716 View
  • 2924 Score
  • 76 Answer
  • Tags:   c++ string split

5 Answered Questions

[SOLVED] How to use range-based for() loop with std::map?

9 Answered Questions

[SOLVED] Pretty-print C++ STL containers

1 Answered Questions

[SOLVED] C++ STL; Iterate over class which contains STL container?

2 Answered Questions

[SOLVED] Preferred standard use: range based for or std::for_each

4 Answered Questions

[SOLVED] C++11 range based loop: get item by value or reference to const

  • 2013-03-02 15:25:07
  • masoud
  • 77936 View
  • 200 Score
  • 4 Answer
  • Tags:   c++ c++11

1 Answered Questions

Sponsored Content