OSDev.org

Posted: **Fri Mar 13, 2009 3:14 am**

Hi,

yesterday I came across some things, which I asked myself how to solve best. But I only came up with a (what I call) crappy solution. So I decided to ask here, because I'm sure someone (e.g. Solar?) can help me.

So here we go... You know the C++ (i)ostreams:

Code: Select all

std::cout << "foo" << 42 << "bar" << std::endl;

And you know the bash pipe and the concept of filters:

Code: Select all

cat someFile | grep fooBar | gawk magicPattern

It looks very similar while it isn't. But the question is: How can I implement such filters with the C++ streams?

Say you have something like

Code: Select all

std::cout << "foo" << f << s;

where f is a "filter" and s is a string/const char*/something << understands...

It seems similar to:

Code: Select all

std::cout << "foo" << hex << 42;

where you can see the manipulator "hex" as a filter.

So... Which is the best way to achieve this? Thanks in advance!

--TS

Posted: **Fri Mar 13, 2009 3:37 am**

Disclaimer: Never dug into the depths of this, and didn't do research first - this is ad-hoc.

----

It all revolves about the leftmost object. In your example, it's std::cout - type std::ostream. That type doesn't know about filters.

But I can imagine you could set up your own ostream derivate. Overload its operator<<() to recognize a certain "filter" type (e.g. MyFilter), storing it in an internal (initially empty) vector. Any input received by other operator<<() calls gets "piped" through any MyFilter objects in the vector.

That's going to be one hell of a mess of operator<<() overloads, there, as MyFilter must be able to handle any types that might happen to appear...

----

Again, this was very much ad-hoc, and I haven't had breakfast yet, so I might be completely off-the-mark. Check if Boost.org has something along these lines.

Posted: **Fri Mar 13, 2009 4:11 am**

First of all, I'd like to say that I'm not quite sure as to what exactly you're trying to achieve here. I'll just go with what I think is right -- that you are attempting to create an effect similar to that of standard Unix shell piping within a C++ program. That is, you are trying to let different "filters" process the input (which can be the output of an earlier filter) one at a time until you reach the last filter (which would be equivalent to the command on the right side of the last piping character, |). I hacked together a quick example that shows how this can be accomplished by having a base class "Filter" which provides a pure virtual "process" function, and a function to connect the filter to another filter. When a filter is connected to another, its output will be sent as input to that filter, and that filter will pass its output on, etc, until the end of the line is reached.

In the example I provide here, I use a filter chain containing two filters: one that converts all alphabetic characters to upper case, and one that strips all digits from the input string. In my main() function, I feed the first filter with a testing string, "Hello World 1234!". This string is first run through the upper case converter filter, which outputs "HELLO WORLD 1234!". Then, it is fed through the digit remover filter, which outputs "HELLO WORLD !". We retrieve the output from the digit remover filter, as it is the last, and print it the regular way.

Note that this is by no means the best possible way of accomplishing this. Far more robust and clean solutions could be created, but it works rather well for a hack.

I hope I got your question right, and that you will find the attached code useful. Good luck!

Code: Select all

#include <iostream>
#include <cctype>

class Filter
{
public:
    Filter()
    {
        m_next = NULL;
        m_output = "";
    }

    virtual ~Filter()
    {
    }

    void connect(Filter *next)
    {
        m_next = next;
    }

    virtual void process(const std::string& input) = 0;

    void next()
    {
        if(m_next)
        {
            m_next->process(m_output);
        }
    }

    const std::string& getOutput() const
    {
        return m_output;
    }

protected:
    std::string m_output;

private:
    Filter *m_next;
};

class UppercaseConverter : public Filter
{
public:
    UppercaseConverter() : Filter()
    {
    }

    virtual ~UppercaseConverter()
    {
    }

    virtual void process(const std::string& input)
    {
        const size_t len = input.size();

        // Clear old output!
        m_output.clear();

        for(size_t i = 0; i < len; i++)
        {
            m_output += toupper(input[i]);
        }

        // Make sure we pass it on.
        this->next();
    }
};

class DigitRemover : public Filter
{
public:
    DigitRemover() : Filter()
    {
    }

    virtual ~DigitRemover()
    {
    }

    virtual void process(const std::string& input)
    {
        const size_t len = input.size();

        // Clear previous outputs.
        m_output.clear();

        for(size_t i = 0; i < len; i++)
        {
            // Include all but digits.
            if(!isdigit(input[i]))
            {
                m_output += input[i];
            }
        }

        // Invoke next filter, if any.
        this->next();
    }
};

int main()
{
    DigitRemover dgr;
    UppercaseConverter ucc;

    ucc.connect(&dgr);
    ucc.process("Hello World 1234!");
    std::cout << "Result: " << dgr.getOutput() << std::endl;

    return 0;
}

Posted: **Fri Mar 13, 2009 4:25 am**

Solar wrote:But I can imagine you could set up your own ostream derivate. Overload its operator<<() to recognize a certain "filter" type (e.g. MyFilter), storing it in an internal (initially empty) vector. Any input received by other operator<<() calls gets "piped" through any MyFilter objects in the vector.

Hm... Nice, I think you imagine the same way as I explored a bit. That was my first quick and very very dirty try:

Code: Select all

#include <iostream>
#include <ostream>
#include <string>

struct Filter {
  std::string filter;
  Filter(std::string s) : filter(s) {}
};
Filter globalfilter("");


std::ostream& operator<< (std::ostream& os, Filter f) {
  globalfilter = f;
  return os;
}

std::ostream& operator<< (Filter f, std::ostream& os) {
  return os;
}

std::ostream& operator<< (std::ostream& os, std::string s) {
  if (s.find(globalfilter.filter) != std::string::npos) os << s.data();
  return os;
}


int main() {
  Filter f("C++");
  Filter g("Java");

  std::string a("C++ rocks");
  std::string b("Java is just for mollycoddles (joke!)");
  std::string c("I love C++!");

  std::cout << f << a << "," << b << "," << c << std::endl;
  std::cout << g << a << "," << b << "," << c << std::endl;

  return 0;
}

That works fine. So far it allows only one filter to be applied, but your vector idea is the obvious extension. I didn't check if there's some memory leaks or something. It was just a shoot from the hip.

What do you think?

That's going to be one hell of a mess of operator<<() overloads, there, as MyFilter must be able to handle any types that might happen to appear...

I agree. Maybe one could use the operator<<() of cout as a guidance?! I think there will mainly occur const char*, string and integer types for output, so it isn't that bad...

Thanks so far!

--TS

Posted: **Fri Mar 13, 2009 5:13 am**

Do you mean like dynamically overloading the << operator at runtime?

I see two ways of doing this:
- Use "operator <<(void *)" and use RTTI to compare it against the types that you can parse. (slow)
- Inherit printable objects from a common interface with a print() function so effectively you're doing:

Code: Select all

Printer &operator << (IPrintable *toPrint)
{
   toPrint->print();   
   return this;
}

Printer &operator << (IPrintable &toPrint)
{
  toPrint.print();
  return this;
}

(fast)

Posted: **Fri Mar 13, 2009 5:50 am**

MessiahAndrw wrote:Do you mean like dynamically overloading the << operator at runtime?

I see two ways of doing this:
- Use "operator <<(void *)" and use RTTI to compare it against the types that you can parse. (slow)
- Inherit printable objects from a common interface with a print() function [...] (fast)

Why at runtime? Is there any (important) case where this is not decidable at compile time? I can't imagine any at the moment.

Your second approach comes close to

Code: Select all

std::ostream& operator<< (std::ostream& os, std::string s)

but is a bit more flexible... So, yes, that would be my preferred way.

BTW, isn't that a wonderful case for templates?!

But that doesn't say anything about the filtering problem... But at least it seems that I was almost on the right way.

--TS

Posted: **Fri Mar 13, 2009 6:02 am**

First thing first: There's no such thing as "the" operator<<() of ostream. In fact, each class has to provide it's own overloading of operator<<() if it wants to be "streamed" correctly:

Code: Select all

class Point
{
    public:
        Point( int x, int y );
    ...
    friend ostream & operator<<( ostream & os, const MyClass & mc );
};

ostream & operator<<( ostream & os, const MyClass & mc )
{
    os << mc.x << ":" << mc.y;
}

See? Each class is responsible for "streaming" itself in a meaningful way by overloading the global operator<<( ostream &, ... ).

Now, the beauty with the idea I outlined above is that your "filtering" ostream derivative should be able to use the global operator<<() of each class through the common base class ostream, i.e. the classes themselves needn't be aware of being "stream-filtered".

Mind, I'm in an awful hurry today and still haven't thought this through. It's just a gut feeling that it should work.

Posted: **Fri Mar 13, 2009 9:28 am**

Solar wrote:First thing first: There's no such thing as "the" operator<<() of ostream. In fact, each class has to provide it's own overloading of operator<<() if it wants to be "streamed" correctly:
Code: Select all
class Point
{
    public:
        Point( int x, int y );
    ...
    friend ostream & operator<<( ostream & os, const MyClass & mc );
};

ostream & operator<<( ostream & os, const MyClass & mc )
{
    os << mc.x << ":" << mc.y;
}
See? Each class is responsible for "streaming" itself in a meaningful way by overloading the global operator<<( ostream &, ... ).

Now, the beauty with the idea I outlined above is that your "filtering" ostream derivative should be able to use the global operator<<() of each class through the common base class ostream, i.e. the classes themselves needn't be aware of being "stream-filtered".

Mind, I'm in an awful hurry today and still haven't thought this through. It's just a gut feeling that it should work.

Solar is right, this is the way C++ does it. If you look at the 'string' header file, you'll see that there are loads of templated (and thus, I believe, filled in at compile-time, not runtime) public operators. If you give an operator an ostream 'ostr' as first argument and an object 'str' of your custom string class as second argument, you can do

Code: Select all

ostr << str;

Supposing you're overloading the '<<' operator. But, even though C++ doesn't do it this way, it is also possible to create overloaded operators inside the class and it will work the same way, you'd do something like:

Code: Select all

struct CustomString {};

class myOstream
{
public:
   myOstream& operator<<(const CustomString& str)
   {
       //Provide implementation to print a string.
       
       //Return a reference to this object, this ensures that the user can switch more than 1 '<<'.
       //like 'myOstreamObj << str1 << str2;'
       return (*this);
   }
};

If I was correct earlier, you can use templates to make code be filled in at compile-time instead of runtime. I could be wrong though.

Posted: **Fri Mar 13, 2009 9:47 am**

My basic idea was to derive from ostream so you could use all those already-implemented operator<<( ostream &, ... ) functions, instead of having to re-implement them.

Your "myOstream" class also cannot access member variables of CustomString (or any other class)... that's why operator<<() is declared "global friend"...

Posted: **Fri Mar 13, 2009 1:44 pm**

Solar wrote:My basic idea was to derive from ostream so you could use all those already-implemented operator<<( ostream &, ... ) functions, instead of having to re-implement them.

Your "myOstream" class also cannot access member variables of CustomString (or any other class)... that's why operator<<() is declared "global friend"...

[Endless C++ Discussion Point Which I Think Is Bull]
But in a GOOD C++ class you should always make mutators and accessors for EVERY variable and hide ALL your member variables! (Cheesy)
[/Endless C++ Discussion Point Which I Think Is Bull]

Posted: **Fri Mar 13, 2009 6:48 pm**

Hi,
Just a small suggesstion , you might want to take a look at iomanip before rolling your own 'filters' .

Regards
Shrek

Posted: **Mon Mar 16, 2009 1:16 pm**

Creature wrote:[Endless C++ Discussion Point Which I Think Is Bull]
But in a GOOD C++ class you should always make mutators and accessors for EVERY variable and hide ALL your member variables! (Cheesy)
[/Endless C++ Discussion Point Which I Think Is Bull]

Don't ever get into the same office with me on business matters, or I will have to shoot you.

OSDev.org

cout and filters

cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters

Re: cout and filters