OSDev.org

Combuster wrote:Re: C++ reading info from a webpage
1: Implement the HTTP protocol to download the page in question
2: Parse the XML to get the wanted data.

Program to an interface not an implementation

dak91 wrote:I made this simple url_get function in C/Linux socket

http://www.inventati.org/dak/src/c/geturl.cpp

fronty wrote:

dak91 wrote:I made this simple url_get function in C/Linux socket

http://www.inventati.org/dak/src/c/geturl.cpp

Did you even try to compile that? It doesn't compile, it isn't written in C and more correct name of the API is Berkeley sockets.

I made this simple version in C, uses getaddrinfo(3) (better version of gethostbyname(3) and getserbyname(3)). Tested on FreeBSD/amd64 and NetBSD/sparc. link

dak91 wrote:

fronty wrote:

dak91 wrote:I made this simple url_get function in C/Linux socket

http://www.inventati.org/dak/src/c/geturl.cpp

Did you even try to compile that? It doesn't compile, it isn't written in C and more correct name of the API is Berkeley sockets.

I made this simple version in C, uses getaddrinfo(3) (better version of gethostbyname(3) and getserbyname(3)). Tested on FreeBSD/amd64 and NetBSD/sparc. link

I made that code 2 years ago, but I remember that it compile correctly...

anyway thanks for the correction about the api name

using namespace std;

	for(int x=0;x<strlen(serverd.c_str());x++){ 

		if(serverd.c_str()[x] == '/'){ 

			y = x; 
			break; 

	if(y!=0){
		data = server = "";
		for(int x=y;x<strlen(serverd.c_str());x++){ data += serverd.c_str()[x]; }
		for(int x=0;x<y;x++){ server += serverd.c_str()[x]; }
	}

	if(connect(y,(struct sockaddr*) &server_addr, sizeof(server_addr))  != 0){
		cout<<"Cannot connect...\n";
		return ""; 
	}

		string get_request = "GET "+data+" HTTP/1.1\n\n\n\n";

		send(y, get_request.c_str(), strlen(get_request.c_str()), 0);

		char dat[10000];
		for(int x=0;x<10000;x++){ dat[x] = '\0'; }

		recv(y, dat, 10000, 0);
		close(y);	

		y = 0;
		char buf;
		while(buf!='\0'){
			buf = dat[y];
			data += buf;
			y++;
		}
		return data;

Owen wrote:Incidentally, I notice that fronty's also makes the "send/recv will always do everything in one go" assumption, but is on the whole at least much cleaner.

Owen wrote: Reading that code...

Code: Select all

using namespace std;

OMG. Bringing in craptonne of unknown crap. Crazy.

Code: Select all

	for(int x=0;x<strlen(serverd.c_str());x++){ 

We've never heard of std::string::length() now? Which is far faster?

Oh, and were executing strlen every freaking time through the loop

Code: Select all

		if(serverd.c_str()[x] == '/'){ 

Wait, so were reimplementing std::string::find/strchr now?

Code: Select all

			y = x; 
			break; 

Again with the one letter variables. I'm glad nobody is trying to read your code.

Oh, wait...

Code: Select all

	if(y!=0){
		data = server = "";
		for(int x=y;x<strlen(serverd.c_str());x++){ data += serverd.c_str()[x]; }
		for(int x=0;x<y;x++){ server += serverd.c_str()[x]; }
	}

Yay! Lets poorly reimplement std::string's substring constructor

Code: Select all

	if(connect(y,(struct sockaddr*) &server_addr, sizeof(server_addr))  != 0){
		cout<<"Cannot connect...\n";
		return ""; 
	}

Oh, I encountered an error. Lets print it. Not, you know, return it to the caller. Not emit it to the error stream either.

Code: Select all

		string get_request = "GET "+data+" HTTP/1.1\n\n\n\n";

Yargh! Lets build an invalid HTTP/1.1 request (For a start, you're missing the required Host: header. And you definitely want that one, too. I mean, it would be such a shame if 99% of websites didn't work.

Oops.

Code: Select all

		send(y, get_request.c_str(), strlen(get_request.c_str()), 0);

Lets pretend that the OS will always send my data in one go.

Oh wait, it won't...

Code: Select all

		char dat[10000];
		for(int x=0;x<10000;x++){ dat[x] = '\0'; }

Wait, we are reinventing memset now?!

And what if my page is bigger than 10kb? Did dynamically allocated buffers go out of style?

I mean, we are just forgetting that std::string and std::stringstream are part of the language?

Code: Select all

		recv(y, dat, 10000, 0);
		close(y);	

Lets pretend the OS will always return the page in one go...

Code: Select all

		y = 0;
		char buf;
		while(buf!='\0'){
			buf = dat[y];
			data += buf;
			y++;
		}
		return data;