By Agusti-N


2008-10-24 18:19:42 8 Comments

Is it possible to serialize and deserialize a class in C++?

I've been using Java for 3 years now, and serialization / deserialization is fairly trivial in that language. Does C++ have similar features? Are there native libraries that handle serialization?

An example would be helpful.

13 comments

@Calmarius 2019-10-09 10:05:57

I'm using the following template to implement serialization:

template <class T, class Mode = void> struct Serializer
{
    template <class OutputCharIterator>
    static void serializeImpl(const T &object, OutputCharIterator &&it)
    {
        object.template serializeThis<Mode>(it);
    }

    template <class InputCharIterator>
    static T deserializeImpl(InputCharIterator &&it, InputCharIterator &&end)
    {
        return T::template deserializeFrom<Mode>(it, end);
    }
};

template <class Mode = void, class T, class OutputCharIterator>
void serialize(const T &object, OutputCharIterator &&it)
{
    Serializer<T, Mode>::serializeImpl(object, it);
}

template <class T, class Mode = void, class InputCharIterator>
T deserialize(InputCharIterator &&it, InputCharIterator &&end)
{
    return Serializer<T, Mode>::deserializeImpl(it, end);
}

template <class Mode = void, class T, class InputCharIterator>
void deserialize(T &result, InputCharIterator &&it, InputCharIterator &&end)
{
    result = Serializer<T, Mode>::deserializeImpl(it, end);
}

Here T is the type you want to serialize Mode is a dummy type to differentiate between different kinds of serialization, eg. the same integer can be serialized as little endian, big endian, varint, etc.

By default the Serializer delegates the task to the object being serialized. For built in types you should make a template specialization of the Serializer.

Convenience function templates are also provided.

For example little endian serialization of unsigned integers:

struct LittleEndianMode
{
};

template <class T>
struct Serializer<
    T, std::enable_if_t<std::is_unsigned<T>::value, LittleEndianMode>>
{
    template <class InputCharIterator>
    static T deserializeImpl(InputCharIterator &&it, InputCharIterator &&end)
    {
        T res = 0;

        for (size_t i = 0; i < sizeof(T); i++)
        {
            if (it == end) break;
            res |= static_cast<T>(*it) << (CHAR_BIT * i);
            it++;
        }

        return res;
    }

    template <class OutputCharIterator>
    static void serializeImpl(T number, OutputCharIterator &&it)
    {
        for (size_t i = 0; i < sizeof(T); i++)
        {
            *it = (number >> (CHAR_BIT * i)) & 0xFF;
            it++;
        }
    }
};

Then to serialize:

std::vector<char> serialized;
uint32_t val = 42;
serialize<LittleEndianMode>(val, std::back_inserter(serialized));

To deserialize:

uint32_t val;
deserialize(val, serialized.begin(), serialized.end());

Due to the abstract iterator logic, it should work with any iterator (eg. stream iterators), pointer, etc.

@Neil McGill 2019-09-07 01:41:00

Here is a simple serializer library I knocked up. It's header only, c11 and has examples for serializing basic types. Here's one for a map to class.

https://github.com/goblinhack/simple-c-plus-plus-serializer

#include "c_plus_plus_serializer.h"

class Custom {
public:
    int a;
    std::string b;
    std::vector c;

    friend std::ostream& operator<<(std::ostream &out, 
                                    Bits my)
    {
        out << bits(my.t.a) << bits(my.t.b) << bits(my.t.c);
        return (out);
    }

    friend std::istream& operator>>(std::istream &in, 
                                    Bits my)
    {
        in >> bits(my.t.a) >> bits(my.t.b) >> bits(my.t.c);
        return (in);
    }

    friend std::ostream& operator<<(std::ostream &out, 
                                    class Custom &my)
    {
        out << "a:" << my.a << " b:" << my.b;

        out << " c:[" << my.c.size() << " elems]:";
        for (auto v : my.c) {
            out << v << " ";
        }
        out << std::endl;

        return (out);
    }
};

static void save_map_key_string_value_custom (const std::string filename)
{
    std::cout << "save to " << filename << std::endl;
    std::ofstream out(filename, std::ios::binary );

    std::map< std::string, class Custom > m;

    auto c1 = Custom();
    c1.a = 1;
    c1.b = "hello";
    std::initializer_list L1 = {"vec-elem1", "vec-elem2"};
    std::vector l1(L1);
    c1.c = l1;

    auto c2 = Custom();
    c2.a = 2;
    c2.b = "there";
    std::initializer_list L2 = {"vec-elem3", "vec-elem4"};
    std::vector l2(L2);
    c2.c = l2;

    m.insert(std::make_pair(std::string("key1"), c1));
    m.insert(std::make_pair(std::string("key2"), c2));

    out << bits(m);
}

static void load_map_key_string_value_custom (const std::string filename)
{
    std::cout << "read from " << filename << std::endl;
    std::ifstream in(filename);

    std::map< std::string, class Custom > m;

    in >> bits(m);
    std::cout << std::endl;

    std::cout << "m = " << m.size() << " list-elems { " << std::endl;
    for (auto i : m) {
        std::cout << "    [" << i.first << "] = " << i.second;
    }
    std::cout << "}" << std::endl;
}

void map_custom_class_example (void)
{
    std::cout << "map key string, value class" << std::endl;
    std::cout << "============================" << std::endl;
    save_map_key_string_value_custom(std::string("map_of_custom_class.bin"));
    load_map_key_string_value_custom(std::string("map_of_custom_class.bin"));
    std::cout << std::endl;
}

Output:

map key string, value class
============================
save to map_of_custom_class.bin
read from map_of_custom_class.bin

m = 2 list-elems {
    [key1] = a:1 b:hello c:[2 elems]:vec-elem1 vec-elem2
    [key2] = a:2 b:there c:[2 elems]:vec-elem3 vec-elem4
}

@Head Geek 2008-10-24 18:24:05

The Boost::serialization library handles this rather elegantly. I've used it in several projects. There's an example program, showing how to use it, here.

The only native way to do it is to use streams. That's essentially all the Boost::serialization library does, it extends the stream method by setting up a framework to write objects to a text-like format and read them from the same format.

For built-in types, or your own types with operator<< and operator>> properly defined, that's fairly simple; see the C++ FAQ for more information.

@Agnel Kurian 2013-07-02 08:19:00

It seems to me that boost::serialization requires the caller to keep track of the order in which objects are written and read. Is that correct? So if there is a change in the order in which two fields are written between versions of a program then we have an incompatibility. Is this right?

@Head Geek 2013-07-03 13:55:37

I believe that's the case, yes.

@0xDEAD BEEF 2013-09-06 12:22:59

Boost serialization failed me when I tried serializing using 32bit app and de-serializing using 64bit version.

@Head Geek 2013-09-07 14:39:22

That would probably be due to the serializing functions, not the Boost::serialization code itself.

@Ela782 2014-12-26 22:23:52

@0xDEADBEEF: That probably happend when using a binary_(i|o)archive, which introduces other "problems" like endian-ness. Try text_(i|o)archive, it's more platform agnostic.

@Tomáš Zato - Reinstate Monica 2016-01-09 19:25:16

Wow until now I thought this is impossible without some heavy macro bullshit, but this really looks easy and fun! And it's even more OOP-style than java...

@Head Geek 2016-01-10 23:57:39

Yeah, I had reason to implement a serialization/deserialization system using C++11 recently, without Boost. It was dead simple, basically just escape whitespace characters other than the space character itself, and put quotation marks around things that contain spaces and read them as single items. I'll post it in my answer above if anyone's interested.

@Andrea 2016-03-01 12:49:48

A specifc framwork/library solution should not be the accepted answer.

@Head Geek 2016-03-01 13:56:59

@Andrea: The Boost library is a special case. Until C++11 was finalized, it was all but impossible to write modern C++ code without it, so it was closer to a secondary STL than a separate library.

@streaver91 2017-12-29 14:29:34

If you want simple and best performance and don't care about backward data compatibility, try HPS, it's lightweight, much faster than Boost, etc, and much easier to use than Protobuf, etc.

Example:

std::vector<int> data({22, 333, -4444});
std::string serialized = hps::serialize_to_string(data);
auto parsed = hps::parse_from_string<std::vector<int>>(serialized);

@M2tM 2013-01-19 11:29:54

Boost::serialization is a great option, but I've encountered a new project: Cereal which I find much more elegant! I highly suggest investigating it.

@Vincent 2014-09-05 01:57:38

Sweet Persist is another one.

It is possible to serialize to and from streams in XML, JSON, Lua, and binary formats.

@Janusz Syf 2019-08-22 14:40:28

That site seems to be down, all I could find was this old repo: github.com/cwbaker/sweet_persist

@Azoth 2014-03-02 01:00:53

I realize this is an old post but it's one of the first that comes up when searching for c++ serialization.

I encourage anyone who has access to C++11 to take a look at cereal, a C++11 header only library for serialization that supports binary, JSON, and XML out of the box. cereal was designed to be easy to extend and use and has a similar syntax to Boost.

@CyberSnoopy 2014-08-05 05:52:18

The good thing about cereal is that unlike boost, it has minimal metadata (almost none). boost::serialization becomes really annoying when every time you open an archive, it writes its lib version to the stream, which makes appending to a file impossible.

@Robert Ramey 2016-11-30 20:37:59

@CyberSnoopy - there is a flag for suppressing this feature when an archive is created - of course you have to remember it when reading the archive as well.

@jbat100 2012-11-29 11:04:10

I recommend using boost serialization as described by other posters. Here is a good detailed tutorial on how to use it which complements the boost tutorials nicely: http://www.ocoudert.com/blog/2011/07/09/a-practical-guide-to-c-serialization/

@Dave 2011-06-04 06:30:22

You can check the amef protocol, an example of C++ encoding in amef would be like,

    //Create a new AMEF object
    AMEFObject *object = new AMEFObject();

    //Add a child string object
    object->addPacket("This is the Automated Message Exchange Format Object property!!","adasd");   

    //Add a child integer object
    object->addPacket(21213);

    //Add a child boolean object
    object->addPacket(true);

    AMEFObject *object2 = new AMEFObject();
    string j = "This is the property of a nested Automated Message Exchange Format Object";
    object2->addPacket(j);
    object2->addPacket(134123);
    object2->addPacket(false);

    //Add a child character object
    object2->addPacket('d');

    //Add a child AMEF Object
    object->addPacket(object2);

    //Encode the AMEF obejct
    string str = new AMEFEncoder()->encode(object,false);

Decoding in java would be like,

    string arr = amef encoded byte array value;
    AMEFDecoder decoder = new AMEFDecoder()
    AMEFObject object1 = AMEFDecoder.decode(arr,true);

The Protocol implementation has codecs for both C++ and Java, the interesting part is it can retain object class representation in the form of name value pairs, I required a similar protocol in my last project, when i incidentally stumbled upon this protocol, i had actually modified the base library according to my requirements. Hope this helps you.

@epatel 2011-04-02 18:48:26

I suggest looking into Abstract factories which is often used as a basis for serialization

I have answered in another SO question about C++ factories. Please see there if a flexible factory is of interest. I try to describe an old way from ET++ to use macros which has worked great for me.

ET++ was a project to port old MacApp to C++ and X11. In the effort of it Eric Gamma etc started to think about Design Patterns. ET++ contained automatic ways for serialization and introspection at runtime.

@Frank Krueger 2008-10-24 18:44:06

Boost is a good suggestion. But if you would like to roll your own, it's not so hard.

Basically you just need a way to build up a graph of objects and then output them to some structured storage format (JSON, XML, YAML, whatever). Building up the graph is as simple as utilizing a marking recursive decent object algorithm and then outputting all the marked objects.

I wrote an article describing a rudimentary (but still powerful) serialization system. You may find it interesting: Using SQLite as an On-disk File Format, Part 2.

@yoav.aviram 2008-10-24 21:04:25

I recommend Google protocol buffers. I had the chance to test drive the library on a new project and it's remarkably easy to use. The library is heavily optimized for performance.

Protobuf is different than other serialization solutions mentioned here in the sense that it does not serialize your objects, but rather generates code for objects that are serialization according to your specification.

@Agnel Kurian 2013-07-02 08:15:09

Have you had experience serializing objects about 10-50MB in size using this? Documentation seems to say that protocol buffers are best suited for objects about an MB in size.

@Frank Krueger 2008-10-24 18:52:49

As far as "built-in" libraries go, the << and >> have been reserved specifically for serialization.

You should override << to output your object to some serialization context (usually an iostream) and >> to read data back from that context. Each object is responsible for outputting its aggregated child objects.

This method works fine so long as your object graph contains no cycles.

If it does, then you will have to use a library to deal with those cycles.

@einpoklum - reinstate Monica 2014-05-07 06:54:52

Surely, that can't be right... the implemented << operators are used to print human-readable text representations of objects, which is very often not what you want for serialization.

@Carcigenicate 2015-05-08 18:31:38

@einpoklum Instead of defining << for the generic ostream, try defining it for a file stream.

@einpoklum - reinstate Monica 2015-05-09 07:15:26

@Carcigenicate: A log file which takes human-readable text is a file stream.

@Carcigenicate 2015-05-09 13:07:48

@einpoklum I'm not quite sure what you mean. Frank's right though, those operators can be used to serialize. I just defined them to serialize/deserialize a vector.

@ShreevatsaR 2018-06-07 23:17:22

I think the catch is here “You should override << to output your object to some serialization context … Each object is responsible for outputting its…” — the question is about how to avoid having to write that out laboriously for each object: how much can the language or libraries help?

Related Questions

Sponsored Content

23 Answered Questions

[SOLVED] What is the "-->" operator in C++?

37 Answered Questions

1 Answered Questions

[SOLVED] The Definitive C++ Book Guide and List

  • 2008-12-23 05:23:56
  • grepsedawk
  • 2249259 View
  • 4246 Score
  • 1 Answer
  • Tags:   c++ c++-faq

18 Answered Questions

[SOLVED] Why should C++ programmers minimize use of 'new'?

24 Answered Questions

[SOLVED] Image Processing: Algorithm Improvement for 'Coca-Cola Can' Recognition

3 Answered Questions

[SOLVED] How can I change property names when serializing with Json.net?

4 Answered Questions

[SOLVED] What does T&& (double ampersand) mean in C++11?

2 Answered Questions

[SOLVED] Understanding passport serialize deserialize

Sponsored Content