{"id":639,"date":"2016-05-10T18:56:42","date_gmt":"2016-05-10T16:56:42","guid":{"rendered":"http:\/\/queen.run.montefiore.ulg.ac.be\/~barbette\/?p=639"},"modified":"2016-05-10T18:56:43","modified_gmt":"2016-05-10T16:56:43","slug":"a-word-about-mode-3","status":"publish","type":"post","link":"https:\/\/perso.uclouvain.be\/tom.barbette\/a-word-about-mode-3\/","title":{"rendered":"A word about mode 3"},"content":{"rendered":"

First, you probably noted that it’s mostly a bonus, start the mode 3 code when everything else is finished.<\/p>\n

We already saw in the first slides how the rx_rings of the e1000 NIC work. Normally you also now the main receive function : e1000_clean_rx_irq.<\/p>\n

There is one struct\u00a0e1000_rx_ring per hardware RX ring. Normaly, there is only one ring (and you should only care about one for step 7).<\/p>\n

I think the fields are pretty well defined (for once…) :<\/p>\n

191<\/a> struct e1000_rx_ring<\/a> {\r\n192<\/a>         \/* pointer to the descriptor ring memory *\/<\/i><\/b>\r\n193<\/a>         void *desc<\/a>;\r\n194<\/a>         \/* physical address of the descriptor ring *\/<\/i><\/b>\r\n195<\/a>         dma_addr_t<\/a> dma<\/a>;\r\n196<\/a>         \/* length of descriptor ring in bytes *\/<\/i><\/b>\r\n197<\/a>         unsigned int size<\/a>;\r\n198<\/a>         \/* number of descriptors in the ring *\/<\/i><\/b>\r\n199<\/a>         unsigned int count<\/a>;\r\n200<\/a>         \/* next descriptor to associate a buffer with *\/<\/i><\/b>\r\n201<\/a>         unsigned int next_to_use;\r\n202<\/a>         \/* next descriptor to check for DD status bit *\/<\/i><\/b>\r\n203<\/a>         unsigned int next_to_clean;\r\n204<\/a>         \/* array of buffer information structs *\/<\/i><\/b>\r\n205<\/a>         struct e1000_rx_buffer<\/a> *buffer_info<\/a>;\r\n206<\/a>         struct sk_buff<\/a> *rx_skb_top;\r\n207<\/a> \r\n208<\/a>         \/* cpu for rx queue *\/<\/i><\/b>\r\n209<\/a>         int cpu<\/a>;\r\n210<\/a> \r\n211<\/a>         u16<\/a> rdh;\r\n212<\/a>         u16<\/a> rdt;\r\n213<\/a> };<\/pre>\n
The desc is a pointer to the memory zone, containing the ring. This memory zone is accessible by the NIC itself through the dma address. The ring is a contiguous zone of e1000_rx_desc structures. Usually you access it with<\/p>\n
E1000_RX_DESC<\/a>(R<\/a>, i<\/a>)<\/pre>\n
where R is the ring pointer and i the index of the descriptor.<\/p>\n
One descriptor i composed of :<\/p>\n
522<\/a> struct e1000_rx_desc<\/a> {\r\n523<\/a>         __le64<\/a> buffer_addr;     \/* Address of the descriptor's data buffer *\/<\/i><\/b>\r\n524<\/a>         __le16<\/a> length<\/a>;          \/* Length of data DMAed into data buffer *\/<\/i><\/b>\r\n525<\/a>         __le16<\/a> csum<\/a>;            \/* Packet checksum *\/<\/i><\/b>\r\n526<\/a>         u8<\/a> status<\/a>;              \/* Descriptor status *\/<\/i><\/b>\r\n527<\/a>         u8<\/a> errors<\/a>;              \/* Descriptor Errors *\/<\/i><\/b>\r\n528<\/a>         __le16<\/a> special<\/a>;\r\n529<\/a> };<\/pre>\n
The buffer_addr variable is a pointer to the physical memory address where a packet can be received. The problem is that the Kernel normal put hte address of a skbuff->data there. But when a packet is received and its content is copied inside that buffer, how to get back the corresponding skbuff? This is why the e1000_rx_ring structure has also a buffer_info pointer. There is exactly as many buffer_info than e1000_rx_desc. \u00a0The buffer info contains all the software-only information that we need to keep about each descriptors, such as the skbuff which will receive the data pointed by buffer_addr.<\/p>\n
Before receiving any packets, all buffer_addr have to be setted ! Or the NIC wouldn’t know where to put the data. This is done in e1000_configure :<\/p>\n
407<\/a>         for (i<\/a> = 0; i<\/a> < adapter<\/a>->num_rx_queues; i<\/a>++) {\r\n408<\/a>                 struct e1000_rx_ring<\/a> *ring<\/a> = &adapter<\/a>->rx_ring<\/a>[i<\/a>];\r\n409<\/a>                 adapter<\/a>->alloc_rx_buf(adapter<\/a>, ring<\/a>,\r\n410<\/a>                                       E1000_DESC_UNUSED<\/a>(ring<\/a>));\r\n411<\/a>         }<\/pre>\n
alloc_rx_buf is a pointer to the function e1000_alloc_rx_buffers (if you don’t use jumbo frame, and you shouldn’t here). \u00a0We see that this functions is called for all rx rings.<\/p>\n
The function e1000_alloc_rx_buffers is defined at line 4561. \u00a0It calls “e1000_alloc_frag” for each buffers between rx_ring->next_to_use (initialized to 0) up to cleaned_count (in configure, the size of the ring is passed, so this is equivalent to a full reset).<\/p>\n
The memory obtained from e1000_alloc_frag is (probably) not accessible by hardware, so we have to map “DMA map” it :<\/p>\n
4613<\/a>                 buffer_info<\/a>->dma<\/a> = dma_map_single<\/a>(&pdev<\/a>->dev<\/a>,\r\n4614<\/a>                                                   data<\/a>,\r\n4615<\/a>                                                   adapter<\/a>->rx_buffer_len,\r\n4616<\/a>                                                   DMA_FROM_DEVICE);<\/pre>\n
You will have to do this with your fastnet buffers !<\/strong><\/p>\n
Then the result in dma is putted inside buffer_addr (line 4650).<\/p>\n
When some packet are received, e1000_clean_rx_irq will make skbuff out of them, and we’ll need to allocate new buffers and put pointers to them inside the ring so next packet can be received. This is done at the end of e1000_clean_rx_irq :<\/p>\n
4481<\/a>         cleaned_count = E1000_DESC_UNUSED<\/a>(rx_ring<\/a>);\r\n4482<\/a>         if (cleaned_count)\r\n4483<\/a>                 adapter<\/a>->alloc_rx_buf(adapter<\/a>, rx_ring<\/a>, cleaned_count);<\/pre>\n
So when the device goes in mode 3 you have to :<\/p>\n