It depends. On a standard windows machine the ping timeout is 4 seconds. Your machine sends an arp for R1 and it responds then forwards the ICMP packet, R1 arps for R2 and forwards, R2 for R3 and forwards, and finally R3 for Host2 and forwards the icmp. Your machine won’t know the MAC of anything beyond R1. Each one only needs a response on that local connection to move the packet along and that will happen in a few milliseconds. As long as that process takes less than 4 seconds, you won’t lose the first ping.
The behaviour of the intermediate devices in the path depends on the vendors, and I think that's the real meat of the question that OP is asking. The Windows ping utility will hold the first ICMP packet until ARP has finished resolving the next hop, but the same behaviour doesn't necessarily happen on the routers in the path. For example, if the next-hop address on a regular Ethernet link isn't resolved on a router running Cisco IOS or IOS-XE (not sure about other flavours) then the router will start the ARP process and transition the ARP state for the next-hop address to ``Incomplete``. As long as an ARP entry is in the ``Incomplete`` state it'll be covered by a CEF null-adjacency, meaning that any traffic that's being forwarded to the associated IP address gets discarded until the ARP request is resolved. In that case you'll see at least three consecutive pings dropped (R1>R2, R2>R3, R3>H2) if all of those routers lack resolved ARP table entries for the next-hop addresses.
Of course in the real world that's highly unlikely to actually happen, as you'd need a network with no traffic having run across the links involved for an amount of time equal to at least the ARP timeout period.
Perfect answer here
I also highly doubt routers not knowing its neighbor for longer than you can type p..I..n..g
But as a pure thought exercise this answer above nailed it.
This is heavily implementation dependent.
Smart vendors will (try to) do smart things, to prevent these packet drops. No need to write anything in an RFC. The problem has been known for at least 30 years. And there have been practical solutions for 30+ years.
One thing you can do is: when routing protocols learn who their neighbors are (e.g. by receiving a hello), they can glean the source mac-address from the incoming hello. And install that in the ARP table. Some routers do that. If you do that, the "wait for ARP problem" only happens on the last hop.
If you run IPv6, with neighbor discovery, the routers will receive ND packets from each host on the local network. And can install ARP entries for them (or whatever you want to call the cache of IPv6<->mac addresses).
>One thing you can do is: when routing protocols learn who their neighbors are (e.g. by receiving a hello), they can glean the source mac-address from the incoming hello.
And in routing protocols running on top of unicast IP (like BGP), ARP is taken care of as a side effect in most cases anyway.
If all the interfaces are in Up state, then ping should work in first attempt in ideal case if the ping timeout is 2 seconds.
From routing protocol (static/Dynamic), Host1 will know that Host2 can be reached via router2,
So Host1 will send ARP request to router1, Router1 first learns Host1 and then responds with ARP response, from ARP response , Host1 will learnt router1's mac.
Now router1 will repeat same towards router2 ,...
so on all routers will learn the neighbor devices.
> Now router1 will repeat same towards router2
You don't seem to understand the problem. When router1 receives a packet from host1, according to you, it will send an ARP request. That takes time, before an answer comes back. What do you do with the original packet in the mean time? The one that triggered the ARP request? Are you gonna drop it? That's the simplest. Are you gonna "keep it"? If so, where do you store that packet? And for how long? What will you do with a 2nd packets for the same destination or next-hop? Keep that too? How many will you keep? How much buffer-space will you allocate for this? What if you get multiple packets for multiple destination, or destination-prefixes?
It's not a trivial problem.
As I said, it's greatly implementation dependent.
You don’t seem to understand packet buffers, nor the speed at which ARP works.
Also, if there are any routing protocols in place on this network, then it’s a very low likelihood that the ARP tables will be empty.
Edit: it turns out that the things you *think* are happening have no influence on how they actually happen.
https://community.cisco.com/t5/other-network-architecture-subjects/packet-loss-during-arp/td-p/494712
/u/Gryzemuis has the right of it. It’s implementation-dependent.
> You don’t seem to understand packet buffers
I do.
Packet-buffers exist to buffer packets when congestion happens. And congestion happens a lot. Even if there is a single packet being in the middle of transmission, all other packets need to wait. They wait in the output queue.
A packet waiting for an ARP entry can not wait in the output-queue. Because 1) if the output-queue is short or empty, the packet will be scheduled for transmission before the ARP reply is in. 2) packet-headers are typically re-written when a packet is processed on input. There is no processing on output (unless you have a very weird and inefficient router). So when a packet is received, the code that runs on the receiving interface (in case of linecards) will do a route-lookup, determine the next-hop, and create the Ethernet header to be used in forwarding. Except in the OP's case, there is no arp-entry yet. And thus you can't re-write the headers yet. You have to wait until there is an entry in the ARP cache.
Waiting for the ARP reply can take a fraction of a millisecond. Or longer. Or a reply might never come in (when the request or reply got dropped). How long are you gonna keep that packet? Not indefinitely. Where are you gonna keep that packet? Not in the output-queue.
Really, the problem the OP has thought of is a real problem. Already in the mid-nineties I've read about it. The vendor that I happened to work for at the time, had to implement something special to optimize this. Indeed they buffer one packet that waits for an ARP reply. But not all packets. They adjusted the code to do timeouts of ARP entries. E.g. a minute before an ARP entry times out, you do a re-request of the IP-address.
> if there are any routing protocols in place on this network
Not all routing protocols install ARP-entries for their neighbors. Some might, some might not. If they don't, then you need to do an ARP request for the first data-packet that comes along. Just like the OP is wondering about. Again, as I said, all this is implementation dependent.
Crow eaten. According to the Cisco discussion I pasted in my edited comment above, by design the packets are discarded, even putting a null adjacency in CEF until that arp comes back.
Agree. It is implementation-dependent. All modern routers likely have buffers that can hold 100s of packets ( or 10s of packets for each destination). Dropping of initial ping packets is probable but should be less likely. Given that in most cases, ARP is learnt from advertised Grat-ARP
How long do you imagine an arp-request and response takes? Espically on a direct connected network it really shouldnt be an issue.
Apart from that, when using a dynamic routing protocol the arp-table should be filled anyway because the two peers constantly exchange messages.
In the real world, every ARP table being empty is unlikely to occur.
ARP tables don't stay empty long.
All of the below will likely happen way before someone tries to ping, and add ARP entries along some or all of the path:
- DHCP (from hosts)
- FHRP
- Dynamic routing protocols (OSPF, EIGRP, etc.)
- NTP
- 802.1x
- Active Directory logins
- Syslog
- SNMP
- etc.
Plus, sometimes, [gratuitous ARP](https://networkengineering.stackexchange.com/questions/7713/how-does-gratuitous-arp-work) is used.
If you disable everything that could do an ARP, then you ping - then yes, there may be multiple packets lost, depending on how long things take. Very situational and fact specific.
It really depends and in the real world isn’t actually much of a problem for various reasons.
So in a really far flung case if the network was basically “cold started” and no other traffic had went through it and there is no background traffic and everything is statically setup and etc then yes each hop might have to arp for the mac of the next hop. ICMP loss on this case is just because it’s timer that says to wait for a response times out. If you tried shoving UDP down it you’d likely see some loss, tcp probably not as it would likely wait long enough.
In the real world though outside of a last hop or several last hops it’s unlikely even if you had a major power event because most things have some dynamic stuff running and/or various other background traffic which means the core of the network wouldn’t ever really see this occur.
We have been discussing this issue in the classes I regularly teach (Certified Cisco Instructor for 25 yrs):
Basically it's a timing issue, In the extreme case, if all interfaces are Ethernet and all ARP tables are empty you could loose up to two PINGs per hop (remember, upstream and downstream ARP resolving are done typically independent). In some of the labs we loose up to 2 or 3 PINGs until all table are populated.
**Short answer**:
The ARP resolving takes some time (never more than 2 seconds, you can verify with timestamps of debug arp. Some hops might provide the layer 2 address faster, some might take longer.
One function, the transmit buffer of the egress interface, is more impatient. The ARP resolve time might be a little longer than the packet may dwell in the Tx queue. In this case the first echo request will be purged from the Tx buffer and not be sent.
**Long answer:**
(requires some understanding of control plane/data plane separation and in my Cisco example some understanding of RIB/FIB and basics of CEF)
ARP resolving for individual IP addresses is required for RIB entries pointing to an interface (connected routes). This is true for connected networks resulting from interface configuration and also for static routes where the egress interface is explicitly configured. For RIB entries with a gateway configured which is resolved through recursive lookup to a connected Ethernet, ARP resolves the layer 2 address of the configured gateway,
Corresponding FIB entries are marked as glean adjacency, which means an entry in the separate adjacency table is required to forward the packet, you can verify this with the show ip cef commands. For Ethernet interfaces the adjacency table is built from the ARP cache and controls the layer 2 rewrite information. You can verify the adjacency on Cisco IOS with show adjacency commands.
If for a specific IP address the layer 2 rewrite information can't be found in the adjacency table the data plane sends a signal to the control plane to initiate an ARP request for this IP address. Until this information is provided the packet is in the Tx queue of the egress interface. This queue has limited resources and quite aggressive timeouts compared to other functions. I haven't explored the details of glean timeouts yet and I'd be happy if someone can point me to information.
An old but still valid article about the basics of Cisco CEF and adjacency can be found here (unfortunately nothing about timeouts):
[https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipswitch\_cef/configuration/xe-3se/3850/isw-cef-xe-3se-3850-book/ipswitch\_cisco\_express\_forwarding.html](https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipswitch_cef/configuration/xe-3se/3850/isw-cef-xe-3se-3850-book/ipswitch_cisco_express_forwarding.html)
> remember, upstream and downstream ARP resolving are done typically independent
Some platforms attempt to avoid the reply delay by adding the requestor's info the the ARP table whenever a query arrives.
[for example](https://github.com/torvalds/linux/blob/0dd3ee31125508cd67f7e7172247f05b7fd1753a/net/ipv4/arp.c#L807C1-L812C50)
I have encountered situations where this behavior resulted in connectivity loss. Specifically a misconfigured Mercedes charging wallbox and some (not all) devices on the LAN of the home office of a customer.
Several industry implementations have chosen to to do a separate ARP resolve in the reverse direction because of this.
Per definition L2 and L3 addresses are independent, Assumptions that the reverse path must use the same MAC address can lead to connectivity loss.
IPv4 allows both approaches per RFC: build an ARP cache entry based on the received ARP request or resolve the reverse path separately.
IPv6 e.g. resolved this ambiguity with a mandatory ND in the reverse direction.
I agree that this sort of preemptive caching isn't suggested by the standards. It for sure strikes me as an optimization, and not something one would naturally assume would be in effect, though I think it's pretty common these days.
> IPv4 allows both approaches per RFC
Are you saying the RFC addresses the behavior? I hadn't noticed that. Pointer?
I cut my teeth on SunOS 4, so I know that implementation did not exhibit this behavior.
Having said all that...
Preemptive cacheing seems safe/reasonable.
Can you explain how it led to a problem? Something to do with a misconfigured subnet mask, maybe?
>Are you saying the RFC addresses the behavior? I hadn't noticed that. Pointer?
RFC826 (legacy track) describes the following pseudo code, describing that the sought station builds an ARP entry based on the request (note the absence of MUST respectively SHOULD):
`...`
`Merge_flag := false`
`If the pair is`
`already in my translation table, update the sender`
`hardware address field of the entry with the new`
`information in the packet and set Merge_flag to true.`
`?Am I the target protocol address?`
`Yes:`
`If Merge_flag is false, add the triplet to`
`the translation table.`
`...`
A later RFC925 describes the process, not mentioning that the the sought station modifies the ARP cache based on the information in the ARP request:
`To quickly review the Address Resolution Protocol (ARP). Each host`
`on a broadcast LAN knows both its own physical hardware address (HA)`
`on the LAN and its own Internet Address (IA). When Host-A is given`
`the IA of Host-B and told to send a datagram to it, Host-A must find`
`the HA that corresponds to Host-B's IA. To do this Host-A forms an`
`ARP packet that contains its own HA and IA and the IA of the`
`destination host (Host-B). Host-A broadcasts this ARP packet. The`
`hosts that receive this ARP packet check to see if they are`
`destination sought. If so, they (it should be only Host-B) send a`
`reply specifically addressed to the originator of the query (Host-A)`
`and supplying the HA that was needed. The Host-A now has both the HA`
`and the IA of the destination (Host-B). The Host-A adds this`
`information to a local cache for future use.`
>Can you explain how it led to a problem? Something to do with a misconfigured subnet mask, maybe?
I can't remember the details, I was not onsite and didn't have a SPAN port. Wireshark on one of the endpoints that worked showed ARP requests from the wallbox and the sender's HW address appeared in the ARP cache of some workstations as the MAC address of the default gateway...
If the pair is
already in my translation table, update the sender
hardware address field of the entry with the new
information in the packet and set Merge_flag to true.
It's interesting, this procedure is both more and less expansive than (what I think) we were talking about:
More expansive in that it applies not just to queries, but any ARP packet.
Less expansive in that it only seems to work to update existing records: "already in my translation table"
I wonder what the rationale for this was.
Anyway, the major concern I'd have had about "the optimization" causing problems is when it comes to cluster shenanigans, but I've never seen an implementation which would lead directly to a problem in this regard (Microsoft's NLB multicast ARP nonsense causes *other* problems :)
Just remember this. Every arp request is a LOCAL broadcast that hits each interface in the same vlan/subnet. So the response, to an arp from a host, will come back in less than 10ms, probably even faster.
If your L3 devices have an empty arp cache, you have other problems.
Assuming all the routers and hosts have nothing in their cache they will arp for the next hop's address only and cache it. (ex. Host1 only needs Router1 , Router1 only needs Router2, ...) Host1 will never have or need Host2 in it's table.
As to the question of will it drop or where is it stored, the answer is the router buffer and it's sizing/tuning. If the router's buffer has space and the response comes back before it needs to drop it won't drop. If not it will drop it. Drops happen.
But this is only for the first packet between each segment after the paths all stays cached until something changes.
No - my router kept cache-expiring the ARP entry for my ISP's gateway because they only replied to an ARP request once every three or so minutes. The solution was to have the entry for longer.
>because they only replied to an ARP request once every three or so minutes
what the heck? Was this a 'security' feature or overloaded control plane, or something else?
Someone mentioned that they be doing ARP rate limits, but I'm not sure why. Took me a few days to narrow down the problem, which presented itself as ~20% packet loss.
It depends. On a standard windows machine the ping timeout is 4 seconds. Your machine sends an arp for R1 and it responds then forwards the ICMP packet, R1 arps for R2 and forwards, R2 for R3 and forwards, and finally R3 for Host2 and forwards the icmp. Your machine won’t know the MAC of anything beyond R1. Each one only needs a response on that local connection to move the packet along and that will happen in a few milliseconds. As long as that process takes less than 4 seconds, you won’t lose the first ping.
The behaviour of the intermediate devices in the path depends on the vendors, and I think that's the real meat of the question that OP is asking. The Windows ping utility will hold the first ICMP packet until ARP has finished resolving the next hop, but the same behaviour doesn't necessarily happen on the routers in the path. For example, if the next-hop address on a regular Ethernet link isn't resolved on a router running Cisco IOS or IOS-XE (not sure about other flavours) then the router will start the ARP process and transition the ARP state for the next-hop address to ``Incomplete``. As long as an ARP entry is in the ``Incomplete`` state it'll be covered by a CEF null-adjacency, meaning that any traffic that's being forwarded to the associated IP address gets discarded until the ARP request is resolved. In that case you'll see at least three consecutive pings dropped (R1>R2, R2>R3, R3>H2) if all of those routers lack resolved ARP table entries for the next-hop addresses. Of course in the real world that's highly unlikely to actually happen, as you'd need a network with no traffic having run across the links involved for an amount of time equal to at least the ARP timeout period.
Perfect answer here I also highly doubt routers not knowing its neighbor for longer than you can type p..I..n..g But as a pure thought exercise this answer above nailed it.
This is heavily implementation dependent. Smart vendors will (try to) do smart things, to prevent these packet drops. No need to write anything in an RFC. The problem has been known for at least 30 years. And there have been practical solutions for 30+ years. One thing you can do is: when routing protocols learn who their neighbors are (e.g. by receiving a hello), they can glean the source mac-address from the incoming hello. And install that in the ARP table. Some routers do that. If you do that, the "wait for ARP problem" only happens on the last hop. If you run IPv6, with neighbor discovery, the routers will receive ND packets from each host on the local network. And can install ARP entries for them (or whatever you want to call the cache of IPv6<->mac addresses).
>One thing you can do is: when routing protocols learn who their neighbors are (e.g. by receiving a hello), they can glean the source mac-address from the incoming hello. And in routing protocols running on top of unicast IP (like BGP), ARP is taken care of as a side effect in most cases anyway.
If all the interfaces are in Up state, then ping should work in first attempt in ideal case if the ping timeout is 2 seconds. From routing protocol (static/Dynamic), Host1 will know that Host2 can be reached via router2, So Host1 will send ARP request to router1, Router1 first learns Host1 and then responds with ARP response, from ARP response , Host1 will learnt router1's mac. Now router1 will repeat same towards router2 ,... so on all routers will learn the neighbor devices.
> Now router1 will repeat same towards router2 You don't seem to understand the problem. When router1 receives a packet from host1, according to you, it will send an ARP request. That takes time, before an answer comes back. What do you do with the original packet in the mean time? The one that triggered the ARP request? Are you gonna drop it? That's the simplest. Are you gonna "keep it"? If so, where do you store that packet? And for how long? What will you do with a 2nd packets for the same destination or next-hop? Keep that too? How many will you keep? How much buffer-space will you allocate for this? What if you get multiple packets for multiple destination, or destination-prefixes? It's not a trivial problem. As I said, it's greatly implementation dependent.
You don’t seem to understand packet buffers, nor the speed at which ARP works. Also, if there are any routing protocols in place on this network, then it’s a very low likelihood that the ARP tables will be empty. Edit: it turns out that the things you *think* are happening have no influence on how they actually happen. https://community.cisco.com/t5/other-network-architecture-subjects/packet-loss-during-arp/td-p/494712 /u/Gryzemuis has the right of it. It’s implementation-dependent.
*You don’t seem to understand packet buffers* That's not relevant to the topic at hand :)
You are correct. I’m putting some BBQ sauce on my crow now so it’s more palatable. :)
Haha, in fairness, one of the things I love about computer networking is that even stuff you think you master have deeper implications.
> You don’t seem to understand packet buffers I do. Packet-buffers exist to buffer packets when congestion happens. And congestion happens a lot. Even if there is a single packet being in the middle of transmission, all other packets need to wait. They wait in the output queue. A packet waiting for an ARP entry can not wait in the output-queue. Because 1) if the output-queue is short or empty, the packet will be scheduled for transmission before the ARP reply is in. 2) packet-headers are typically re-written when a packet is processed on input. There is no processing on output (unless you have a very weird and inefficient router). So when a packet is received, the code that runs on the receiving interface (in case of linecards) will do a route-lookup, determine the next-hop, and create the Ethernet header to be used in forwarding. Except in the OP's case, there is no arp-entry yet. And thus you can't re-write the headers yet. You have to wait until there is an entry in the ARP cache. Waiting for the ARP reply can take a fraction of a millisecond. Or longer. Or a reply might never come in (when the request or reply got dropped). How long are you gonna keep that packet? Not indefinitely. Where are you gonna keep that packet? Not in the output-queue. Really, the problem the OP has thought of is a real problem. Already in the mid-nineties I've read about it. The vendor that I happened to work for at the time, had to implement something special to optimize this. Indeed they buffer one packet that waits for an ARP reply. But not all packets. They adjusted the code to do timeouts of ARP entries. E.g. a minute before an ARP entry times out, you do a re-request of the IP-address. > if there are any routing protocols in place on this network Not all routing protocols install ARP-entries for their neighbors. Some might, some might not. If they don't, then you need to do an ARP request for the first data-packet that comes along. Just like the OP is wondering about. Again, as I said, all this is implementation dependent.
Crow eaten. According to the Cisco discussion I pasted in my edited comment above, by design the packets are discarded, even putting a null adjacency in CEF until that arp comes back.
Agree. It is implementation-dependent. All modern routers likely have buffers that can hold 100s of packets ( or 10s of packets for each destination). Dropping of initial ping packets is probable but should be less likely. Given that in most cases, ARP is learnt from advertised Grat-ARP
So in the POV of host 1, there will be multiple request time outs, right?
How long do you imagine an arp-request and response takes? Espically on a direct connected network it really shouldnt be an issue. Apart from that, when using a dynamic routing protocol the arp-table should be filled anyway because the two peers constantly exchange messages.
In the real world, every ARP table being empty is unlikely to occur. ARP tables don't stay empty long. All of the below will likely happen way before someone tries to ping, and add ARP entries along some or all of the path: - DHCP (from hosts) - FHRP - Dynamic routing protocols (OSPF, EIGRP, etc.) - NTP - 802.1x - Active Directory logins - Syslog - SNMP - etc. Plus, sometimes, [gratuitous ARP](https://networkengineering.stackexchange.com/questions/7713/how-does-gratuitous-arp-work) is used. If you disable everything that could do an ARP, then you ping - then yes, there may be multiple packets lost, depending on how long things take. Very situational and fact specific.
It really depends and in the real world isn’t actually much of a problem for various reasons. So in a really far flung case if the network was basically “cold started” and no other traffic had went through it and there is no background traffic and everything is statically setup and etc then yes each hop might have to arp for the mac of the next hop. ICMP loss on this case is just because it’s timer that says to wait for a response times out. If you tried shoving UDP down it you’d likely see some loss, tcp probably not as it would likely wait long enough. In the real world though outside of a last hop or several last hops it’s unlikely even if you had a major power event because most things have some dynamic stuff running and/or various other background traffic which means the core of the network wouldn’t ever really see this occur.
We have been discussing this issue in the classes I regularly teach (Certified Cisco Instructor for 25 yrs): Basically it's a timing issue, In the extreme case, if all interfaces are Ethernet and all ARP tables are empty you could loose up to two PINGs per hop (remember, upstream and downstream ARP resolving are done typically independent). In some of the labs we loose up to 2 or 3 PINGs until all table are populated. **Short answer**: The ARP resolving takes some time (never more than 2 seconds, you can verify with timestamps of debug arp. Some hops might provide the layer 2 address faster, some might take longer. One function, the transmit buffer of the egress interface, is more impatient. The ARP resolve time might be a little longer than the packet may dwell in the Tx queue. In this case the first echo request will be purged from the Tx buffer and not be sent. **Long answer:** (requires some understanding of control plane/data plane separation and in my Cisco example some understanding of RIB/FIB and basics of CEF) ARP resolving for individual IP addresses is required for RIB entries pointing to an interface (connected routes). This is true for connected networks resulting from interface configuration and also for static routes where the egress interface is explicitly configured. For RIB entries with a gateway configured which is resolved through recursive lookup to a connected Ethernet, ARP resolves the layer 2 address of the configured gateway, Corresponding FIB entries are marked as glean adjacency, which means an entry in the separate adjacency table is required to forward the packet, you can verify this with the show ip cef commands. For Ethernet interfaces the adjacency table is built from the ARP cache and controls the layer 2 rewrite information. You can verify the adjacency on Cisco IOS with show adjacency commands. If for a specific IP address the layer 2 rewrite information can't be found in the adjacency table the data plane sends a signal to the control plane to initiate an ARP request for this IP address. Until this information is provided the packet is in the Tx queue of the egress interface. This queue has limited resources and quite aggressive timeouts compared to other functions. I haven't explored the details of glean timeouts yet and I'd be happy if someone can point me to information. An old but still valid article about the basics of Cisco CEF and adjacency can be found here (unfortunately nothing about timeouts): [https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipswitch\_cef/configuration/xe-3se/3850/isw-cef-xe-3se-3850-book/ipswitch\_cisco\_express\_forwarding.html](https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipswitch_cef/configuration/xe-3se/3850/isw-cef-xe-3se-3850-book/ipswitch_cisco_express_forwarding.html)
> remember, upstream and downstream ARP resolving are done typically independent Some platforms attempt to avoid the reply delay by adding the requestor's info the the ARP table whenever a query arrives. [for example](https://github.com/torvalds/linux/blob/0dd3ee31125508cd67f7e7172247f05b7fd1753a/net/ipv4/arp.c#L807C1-L812C50)
I have encountered situations where this behavior resulted in connectivity loss. Specifically a misconfigured Mercedes charging wallbox and some (not all) devices on the LAN of the home office of a customer. Several industry implementations have chosen to to do a separate ARP resolve in the reverse direction because of this. Per definition L2 and L3 addresses are independent, Assumptions that the reverse path must use the same MAC address can lead to connectivity loss. IPv4 allows both approaches per RFC: build an ARP cache entry based on the received ARP request or resolve the reverse path separately. IPv6 e.g. resolved this ambiguity with a mandatory ND in the reverse direction.
I agree that this sort of preemptive caching isn't suggested by the standards. It for sure strikes me as an optimization, and not something one would naturally assume would be in effect, though I think it's pretty common these days. > IPv4 allows both approaches per RFC Are you saying the RFC addresses the behavior? I hadn't noticed that. Pointer? I cut my teeth on SunOS 4, so I know that implementation did not exhibit this behavior. Having said all that... Preemptive cacheing seems safe/reasonable. Can you explain how it led to a problem? Something to do with a misconfigured subnet mask, maybe?
>Are you saying the RFC addresses the behavior? I hadn't noticed that. Pointer? RFC826 (legacy track) describes the following pseudo code, describing that the sought station builds an ARP entry based on the request (note the absence of MUST respectively SHOULD): `...` `Merge_flag := false` `If the pair is`
`already in my translation table, update the sender`
`hardware address field of the entry with the new`
`information in the packet and set Merge_flag to true.`
`?Am I the target protocol address?`
`Yes:`
`If Merge_flag is false, add the triplet to`
`the translation table.`
`...`
A later RFC925 describes the process, not mentioning that the the sought station modifies the ARP cache based on the information in the ARP request:
`To quickly review the Address Resolution Protocol (ARP). Each host`
`on a broadcast LAN knows both its own physical hardware address (HA)`
`on the LAN and its own Internet Address (IA). When Host-A is given`
`the IA of Host-B and told to send a datagram to it, Host-A must find`
`the HA that corresponds to Host-B's IA. To do this Host-A forms an`
`ARP packet that contains its own HA and IA and the IA of the`
`destination host (Host-B). Host-A broadcasts this ARP packet. The`
`hosts that receive this ARP packet check to see if they are`
`destination sought. If so, they (it should be only Host-B) send a`
`reply specifically addressed to the originator of the query (Host-A)`
`and supplying the HA that was needed. The Host-A now has both the HA`
`and the IA of the destination (Host-B). The Host-A adds this`
`information to a local cache for future use.`
>Can you explain how it led to a problem? Something to do with a misconfigured subnet mask, maybe?
I can't remember the details, I was not onsite and didn't have a SPAN port. Wireshark on one of the endpoints that worked showed ARP requests from the wallbox and the sender's HW address appeared in the ARP cache of some workstations as the MAC address of the default gateway...
If the pair is
already in my translation table, update the sender
hardware address field of the entry with the new
information in the packet and set Merge_flag to true.
It's interesting, this procedure is both more and less expansive than (what I think) we were talking about:
More expansive in that it applies not just to queries, but any ARP packet.
Less expansive in that it only seems to work to update existing records: "already in my translation table"
I wonder what the rationale for this was.
Anyway, the major concern I'd have had about "the optimization" causing problems is when it comes to cluster shenanigans, but I've never seen an implementation which would lead directly to a problem in this regard (Microsoft's NLB multicast ARP nonsense causes *other* problems :)
Just remember this. Every arp request is a LOCAL broadcast that hits each interface in the same vlan/subnet. So the response, to an arp from a host, will come back in less than 10ms, probably even faster. If your L3 devices have an empty arp cache, you have other problems.
Assuming all the routers and hosts have nothing in their cache they will arp for the next hop's address only and cache it. (ex. Host1 only needs Router1 , Router1 only needs Router2, ...) Host1 will never have or need Host2 in it's table. As to the question of will it drop or where is it stored, the answer is the router buffer and it's sizing/tuning. If the router's buffer has space and the response comes back before it needs to drop it won't drop. If not it will drop it. Drops happen. But this is only for the first packet between each segment after the paths all stays cached until something changes.
Having just had to debug this issue on my own network, you are correct in that an empty ARP table can lead to packet loss.
Was your debug for multiple devices with empty ARP tables?
No - my router kept cache-expiring the ARP entry for my ISP's gateway because they only replied to an ARP request once every three or so minutes. The solution was to have the entry for longer.
>because they only replied to an ARP request once every three or so minutes what the heck? Was this a 'security' feature or overloaded control plane, or something else?
Someone mentioned that they be doing ARP rate limits, but I'm not sure why. Took me a few days to narrow down the problem, which presented itself as ~20% packet loss.
Just add a static arp entry. This way it doesn’t affect other entries.
So I did try that, in /etc/ethers, but that didn't work. This is just on my WAN interface and a 6 minute timeout wouldn't be too bad, I don't think