Page 1 of 2
Guide on how to test network of OS with Qemu ?
Posted: Mon Jul 04, 2016 2:26 am
by wichtounet
Hi,
My next goal for my OS is to have networking working. I have been able to add a network interface to Qemu:
-net nic,vlan=0,model=rtl8139
and from this, I have been able to detect the PCI Configuration and obtain the MAC Address. Now, I want to see if packets are getting received, but I don't see how a) to configure Qemu for this and b) to send packets to the guest debug it.
I know that there are a lot of posts on this subjects and I've a tried a lot of commands for Qemu for bridge or tap, but none of them worked. I ended up with operation not permitted errors, obscure errors (256?) or simply not having network on the host anymore. Moreover, all the posts that I have imply being on Ubuntu, which I'm not (Gentoo), so I need standard commands rather Ubuntu network configuration.
Does someone has a comprehensive guide on how to do this ? Or a good link ?
Thanks
Re: Guide on how to test network of OS with Qemu ?
Posted: Mon Jul 04, 2016 2:56 am
by Kevin
Well, you definitely need some network backend, otherwise your packets go to nowhere. The most powerful version is indeed using a tap device, but it's notoriously tricky to set up as you already noticed. In theory, it's easy: Set up a tap device that is owned by your user (with tunctl or ip), optionally include it in a bridge with your physical network card (brctl) and then give it to qemu.
Other options include "-net user", which emulates a virtual router that NATs you to the host network (by default, the router is 10.0.2.2). The disadvantage here is that you can't simply attach Wireshark to the device. However, with "-net dump" you can still capture the traffic in your virtual network. Maybe this is good enough for you.
Re: Guide on how to test network of OS with Qemu ?
Posted: Mon Jul 04, 2016 3:19 am
by iansjack
Use the -net dump option to dump packets to a file.
Re: Guide on how to test network of OS with Qemu ?
Posted: Mon Jul 04, 2016 4:03 pm
by wichtounet
Thanks guys.
Kevin wrote:Well, you definitely need some network backend, otherwise your packets go to nowhere. The most powerful version is indeed using a tap device, but it's notoriously tricky to set up as you already noticed. In theory, it's easy: Set up a tap device that is owned by your user (with tunctl or ip), optionally include it in a bridge with your physical network card (brctl) and then give it to qemu.
I've finally been able to configure it
Here is the final qemu command I'm using:
Code: Select all
sudo qemu-system-x86_64 -enable-kvm -cpu host -serial file:virtual.log -netdev tap,helper=/usr/libexec/qemu-bridge-helper,id=thor_net0 -device rtl8139,netdev=thor_net0,id=thor_nic0 -vga std -hda hdd.img &
The network relevant part being:
Code: Select all
-netdev tap,helper=/usr/libexec/qemu-bridge-helper,id=thor_net0 -device rtl8139,netdev=thor_net0,id=thor_nic0
I've discovered the bridge helper from Qemu:
http://wiki.qemu.org/Features/HelperNetworking It does almost everything
The only thing necessary is to create the bridge:
To send a packet, I'm using
Now I have to find a tool to generate some packets for me on a specific interface.
Kevin wrote:Other options include "-net user", which emulates a virtual router that NATs you to the host network (by default, the router is 10.0.2.2). The disadvantage here is that you can't simply attach Wireshark to the device. However, with "-net dump" you can still capture the traffic in your virtual network. Maybe this is good enough for you.
I've checked documentation on "user networking" mode, but the problem is that it only supports TCP and UDP and I wanted to start with ICMP, but I'll maybe start to investigate this further.
iansjack wrote:Use the -net dump option to dump packets to a file.
My problem is more to send a packet to the VM right now, but thanks
Re: Guide on how to test network of OS with Qemu ?
Posted: Tue Jul 05, 2016 3:02 am
by Kevin
wichtounet wrote:Now I have to find a tool to generate some packets for me on a specific interface.
Well, with a correctly working bridge, all packets are by definition visible on all interfaces that are connected in this bridge.
Edit: Oh, you may not be including your physical network card in the bridge, I don't think qemu-bridge-helper does that. So you can either do that and effectively make your VM part of the physical network (brctl add; and then I seem to remember that you need to clear the IP address on your ethernet interface and get one for the bridge interface instead); or you leave your physical network alone, assign the bridge an IP address and subnet so that this specific subnet is routed to the bridge (with only a single tap device in it), and everything else still goes to your physical interface. If you later want to access the internet from your VM, you could do normal NAT.
Re: Guide on how to test network of OS with Qemu ?
Posted: Wed Jul 06, 2016 12:34 am
by Boris
Hi,
If you want to generate packets, try looking at the man page for packet(7)
You will want to use AF_PACKET to test raw Ethernet stuff.
Once you have arp working, use SOCK_RAW to generate raw IP packets.
Re: Guide on how to test network of OS with Qemu ?
Posted: Sun Jul 10, 2016 4:11 am
by wichtounet
It seems I'm still not there at all finally
If I ping on tap0, the only thing I'm receiving is ARP request. Even if I reply, I dont' receive any ICMP packets
I also always receive some IPV6 packets at random intervals for some reason.
Kevin wrote:wichtounet wrote:Now I have to find a tool to generate some packets for me on a specific interface.
Well, with a correctly working bridge, all packets are by definition visible on all interfaces that are connected in this bridge.
Edit: Oh, you may not be including your physical network card in the bridge, I don't think qemu-bridge-helper does that. So you can either do that and effectively make your VM part of the physical network (brctl add; and then I seem to remember that you need to clear the IP address on your ethernet interface and get one for the bridge interface instead); or you leave your physical network alone, assign the bridge an IP address and subnet so that this specific subnet is routed to the bridge (with only a single tap device in it), and everything else still goes to your physical interface. If you later want to access the internet from your VM, you could do normal NAT.
No, I did not add my physical card to the bridge. When I do, it does not seem to pass all packets anyway and when I do, I don't have internet anymore on the host computer :s
If I assign an IP to the bridge or to tap0 and put it up, ping indicates that the interface is up, but no packet is transferred to the guest...
Another issue that I seem to have is that the MAC address I read on the interface card is neither that of the tap0 neither of the bridge. Shouldn't that be the same mac as the tap0 interface ?
It seems to me that network testing an OS is harded than to code the network stack itself
Boris wrote:Hi,
If you want to generate packets, try looking at the man page for packet(7)
You will want to use AF_PACKET to test raw Ethernet stuff.
Once you have arp working, use SOCK_RAW to generate raw IP packets.
Thanks, I'll take a look
Re: Guide on how to test network of OS with Qemu ?
Posted: Sun Jul 10, 2016 7:59 am
by Kevin
wichtounet wrote:If I ping on tap0, the only thing I'm receiving is ARP request. Even if I reply, I dont' receive any ICMP packets
Did you check (e.g. with Wireshark) that your ARP reply is indeed correct?
No, I did not add my physical card to the bridge. When I do, it does not seem to pass all packets anyway and when I do, I don't have internet anymore on the host computer :s
That's obviously not what should happen. Did you remove the IP address from your physical interface as I said? When I configured things incorrectly, I occasionally lost my network connection, too, but once configured correctly, it definitely works. And I seem to remember that "correct configuration" means that the physical interface and the tap interface don't have an IP assigned, but the bridge has.
Another issue that I seem to have is that the MAC address I read on the interface card is neither that of the tap0 neither of the bridge. Shouldn't that be the same mac as the tap0 interface ?
No, the tap0 interface is a virtual network device of your host. If you don't bridge it to the physical network, this MAC address is what your VM has to use to send something to the host. The guest MAC address is different and can be configured as an option to -net in qemu.
Re: Guide on how to test network of OS with Qemu ?
Posted: Sun Jul 10, 2016 10:04 am
by wichtounet
Thanks Kevin!
Kevin wrote:wichtounet wrote:If I ping on tap0, the only thing I'm receiving is ARP request. Even if I reply, I dont' receive any ICMP packets
Did you check (e.g. with Wireshark) that your ARP reply is indeed correct?
It seems fine in Wireshark:
Code: Select all
1 0.000000000 fe:48:e4:24:85:ce Broadcast ARP 42 Who has 64.65.66.67? Tell 192.168.20.201
2 0.045463542 RealtekU_12:34:56 fe:48:e4:24:85:ce ARP 42 64.65.66.67 is at 52:54:00:12:34:56
But it is done three times, so there must be something wrong somewhere.
Kevin wrote:No, I did not add my physical card to the bridge. When I do, it does not seem to pass all packets anyway and when I do, I don't have internet anymore on the host computer :s
That's obviously not what should happen. Did you remove the IP address from your physical interface as I said? When I configured things incorrectly, I occasionally lost my network connection, too, but once configured correctly, it definitely works. And I seem to remember that "correct configuration" means that the physical interface and the tap interface don't have an IP assigned, but the bridge has.
I tried, but in my case, I did not have have any internet when I did that. There must be a configuration error somewhere, I'll check again later.
Kevin wrote:Another issue that I seem to have is that the MAC address I read on the interface card is neither that of the tap0 neither of the bridge. Shouldn't that be the same mac as the tap0 interface ?
No, the tap0 interface is a virtual network device of your host. If you don't bridge it to the physical network, this MAC address is what your VM has to use to send something to the host. The guest MAC address is different and can be configured as an option to -net in qemu.
Ok, that works, at least
Re: Guide on how to test network of OS with Qemu ?
Posted: Tue Oct 04, 2016 10:57 am
by wichtounet
I'm still working on network support and I now have a good support (IP,TCP/DNS,HTTP,DHCP...).
There is still one thing I haven't been able to figure out is how to have several machines in the same 'user' network. If I create the first virtual machine with a "socket listen" and then the second virtual machine with "socket connect", I have a lot of weird TCP packets in Wireshark and I cannot make sense of them (maybe they come from me but I don't see how).
Does someone has a good solution for this ?
Re: Guide on how to test network of OS with Qemu ?
Posted: Tue Oct 04, 2016 12:08 pm
by mariuszp
if you can show a screenshot of wireshark or give the CAP file we might be able to help.
Re: Guide on how to test network of OS with Qemu ?
Posted: Tue Oct 04, 2016 11:49 pm
by wichtounet
mariuszp wrote:if you can show a screenshot of wireshark or give the CAP file we might be able to help.
Sure, I'll do that once I'm back at home.
Re: Guide on how to test network of OS with Qemu ?
Posted: Wed Oct 05, 2016 11:55 am
by wichtounet
Here is the pcap file:
https://baptiste-wicht.ch/thor.pcap
The RST packets are not from me and the first FIN,ACK packets neither. Pretty painful when I'm debugging TCP.
Re: Guide on how to test network of OS with Qemu ?
Posted: Fri Oct 07, 2016 4:01 pm
by mariuszp
wichtounet wrote:Here is the pcap file:
https://baptiste-wicht.ch/thor.pcap
The RST packets are not from me and the first FIN,ACK packets neither. Pretty painful when I'm debugging TCP.
1) are you sure one of those addresses isn't QEMU's DHCP server?
2) is some form of NAT involved?
3) is there a firewall involved?
Re: Guide on how to test network of OS with Qemu ?
Posted: Fri Oct 07, 2016 4:03 pm
by mariuszp
In fact, is your TCP segment structure correct? Perhaps you are accidentally setting the wrong bit.