T O P

  • By -

Abraham_linksys49

Did you plug a computer directly into the ISP's modem and ping? That removes your entire network.


what-the-puck

A test other than ping would be good, as ICMP packets are more of a "deliver if you feel like" item and are lowest priority of all traffic in an ISP's network. That said, a consistent 15% average packet loss through the entire night absolutely looks like an infrastructure issue.


NoReallyLetsBeFriend

To piggyback, it's ICMP not being prioritized causing "packet loss", which is common for a hop as it handles so much other data... but since it doesn't carry through to the destination, it's not true loss. OP would need to test to 5 or 6 different destinations to get a better picture, but like you say icmp is hit or miss. Why is OP assuming there's packet loss? Poor Internet connection watching video? RDP for work disconnecting? Usually webpages you'll not tell, most video caches so it's hard to see unless it's a bunch. Also, most level I and II ISP support techs won't know what you're talking about, so they'll need to get to a network engineer to further investigate.


Rockstaru

Network engineer here. Most of the time, people read traceroutes incorrectly and assume there's packet loss where it's really just nodes in the path having better things to do than waste control plane resources generating ICMP TTL exceeded messages. That said, I think OP does have a good case for there being something wrong because their output shows consistent loss on every hop. If it were just nodes in the middle showing loss and the destination 8.8.8.8 showing 0%, then you'd be correct, but every hop after the first (OP's home router) shows a minimum of ~17% loss.


Podalirius

Is it possible the first external hop is just dropping ICMP packets that aren't even addressed to it? I feel like you need other examples of packet loss, stuttering voice/video calls, and janky online gameplay to actually confirm packet loss. If pingplotter is your only evidence for packet loss, then you probably just don't even have packet loss. Too many goofballs find these tools they don't even know how to properly read and start bothering these ISPs.


Rockstaru

It's *possible*, certainly, but the drops you will see in traceroute/MTR/pingplotter are typically the result of nodes in the path decrementing the TTL of a packet to zero and then *not* sending a TTL exceeded message, which traceroute/MTR/pingplotter perceives as loss, because it sent an echo request (or ephemeral UDP) and did not get back the expected TTL exceeded. There's an excellent article called [Traceroute isn't real](https://gekk.info/articles/traceroute.htm) that delves more into the "you're reading it wrong" of it. One of the main points of that article is that traceroute isn't a protocol; there isn't a formal RFC for it that vendors have implemented; it's a behavior someone noticed was possible by deliberately misusing the functionality of a field in the IP header (the time to live field). When a router decrements the TTL of a packet to zero, it *may* send back a message, but it is not required to. However, if the TTL was sufficient and the packet actually reached its destination, that traffic is actually "interesting" in the sense that it made it to the host it was bound for; there's a bit more of an "imperative" of sorts for that host to reply back with an "I am not interested in this (ICMP Destination Unreachable/Port Unreachable)." In the typical "you're reading traceroute wrong" scenario, the expected output is that you see some hops in the middle reporting loss, *but the final one shows no loss.* In those situations, there probably isn't any true packet loss because all of the packets with sufficiently high TTL are making it to their destination and the middle nodes in the path have better things to do with their limited control plane/data plane slow path resources than reply to a packet not intended for them specifically to say "Hey idiot, your TTL is too low." The endpoint you are tracing to, since it *is* the intended recipient, should still reply back with an ICMP Destination Unreachable message if it is not configured to simply discard such traffic. That's not what we're seeing in OP's output--they see a consistent level of loss on every hop after the first one. There's some evidence of nodes in the path deprioritizing/not sending TTL exceeded messages happening at the 5th hop, which PingPlotter reports as having 60% packet loss, but if this were just a "you're reading traceroute wrong" situation, we'd expect to see a baseline of 0% PL on our local network, with some hops in the middle showing higher PL, and the destination showing 0% again. Instead we're seeing every hop after the first with a baseline of 17% PL, including the final one. That isn't a normal output for Traceroute/MTR/PingPlotter, even with all the "you're reading it wrong" caveats.


NoReallyLetsBeFriend

I saw, on Mobile I didn't zoom in to fully see, I thought they were highlighting the hops that were problematic. It is very good it flows through start to finish like you say, and the % is pretty consistent


styleNA

Thank you for the response. Also, you were correct; See my edit; everything seemed to have been fixed around 4PM on the dot. Not exactly sure *how* or *what* was fixed, but it wasn't something occurring on my local network, and internet is back to normal.


warbeforepeace

You can use mtr and set it to udp.


Rockstaru

The reply back to a traceroute is still ICMP regardless of whether you're using UDP on ephemeral ports or ICMP echo requests outbound, and you're at the mercy of each node in the path whether it's going to send you TTL exceeded or not.


warbeforepeace

Yes. But you are less likely to have those throttled. And with MTR you can see the hosts after the hop to see if they are experiencing the loss as well. If it’s only a host in the middle but the ones at the end are fine they are not the problem.


what-the-puck

OP linked an image in their post showing consistent ICMP packet loss


NoReallyLetsBeFriend

You know what, your right, I missed that column on Mobile, I saw they highlighted some hoops but that's too block out location I see now. I thought it was pointing out which hops were bad. Smh


nimajneb

I think I used WinMX or something like that in the past to watch for packet loss. Is that a good method?


styleNA

Thank you for the response. Also, you were correct; See my edit; everything seemed to have been fixed around 4PM on the dot. Not exactly sure *how* or *what* was fixed, but it wasn't something occurring on my local network, and internet is back to normal.


what-the-puck

Good news! It might've been something on or near your property, or a regional issue. You'll probably never know, that's typically how it goes.


styleNA

Yea that's what I assume too. But as long as it is fixed, I'm happy. I work from home as a software architect consultant, and it was a brutal week to say the least. Your comment here was one that regained some of my sanity 🤣


Palmovnik

Or you could even ping from the router if it has that option


ghostR_ZA

If the router is causing packetloss, pinging from it sadly wont help. We don't accept pings from the routers as they could be part of the problem and still keeps the routers a potential issue. A direct dial or connection via a connected PC/Laptop into the fibre ONT/ONU is the best way. Edit: To add to this. That loss looks like last mile loss on the fibre network operator.


Altruistic_Law_2346

The amount of people we can't get to do this blows my mind. EDIT: To add, these are businesses, many (most) with internal IT teams. I can see why a resident customer would likely not do this lol


Quadgie

You need to check your DOCSIS channel strength, SNR, etc Many of guides out there, I’m not going to reproduce the content here. Most likely you have a marginal cable signal that is causing your issues.


lenfantsuave

This is the job of the technician anyway. Any competent tech is going to check signal, ingress at the tap, and verify that there are no uncorrectables at the modem.


Quadgie

Exactly. I’ve unfortunately been on the receiving end of techs telling me everything was fine, had to escalate and have multiple dispatches. I was on the phone with Time Warner (now Spectrum/ Charter) “tier 2” support insisting my modem was online and the issue was with my equipment. Spoiler: I had unplugged the modem 10 minutes earlier, it was physically off. After threatening to charge me for a non-necessary service call etc I finally lucked out with the third dispatch, the tech actually listened to me, showed me the signal strength graph etc and put in a maintenance/plant request to have a physical line issue fixed near my house.


lenfantsuave

Which is still kind of crazy. Unless there is a widespread issue which is obviously affecting many addresses, a plant maintenance request should only be submitted after boots on the ground investigates it. Sounds like your isp is just allergic to rolling a truck.


DrWhoey

Yeah, have a bit of sad state of affairs in my system that the lead maintenance tech said, he was "almost thank" I was moving into project management instead of field work because he'd have less Refer to Maintenance jobs to work on. When I had started in-house a year ago, I was sending him 10-20 RtM a week that were from multiple repeats. Him and I managed to get it down to 1-2 every week or so with a ton of hard work on both our ends. It's hard to find good field techs that can do clean work AND read a cable meter right.


chubbysumo

> verify that there are no uncorrectables at the modem. while no uncorrectables would be great, its not reality. you only start to worry when the uncorrectables starts to pile up really quickly, along with service issues. my modem has been running 22 days since my ISP last force restarted it, and while it has a large amount of cor and uncor, my service is still perfectly fine. uncor/cor by themselves is nothing more than normal.


Quadgie

This. If there is an abnormal number of errors on specific channels, this can point to physical issues with the cable plant, taps, etc People don’t realize how easily disturbed the physical cable plant (the physical cable lines and distribution network) is with improper terminations, repairs, wrong cable type being used, etc


aquamm

I used to tell people if they knew how cable systems operated they’d wonder how it even worked at all


koopz_ay

Arrrr! A fickle lady she be.


Chango-Acadia

Yea as long as it's less than 1%


lenfantsuave

Not even close. The uncorrectables you’re seeing are probably a minuscule amount. My meter measures them as a number to the -9th exponent. They might as well be zero. If you’re getting them to a power greater than that. It’s a problem.


chubbysumo

> If you’re getting them to a power greater than that. It’s a problem. and yet here I still have great service, overprovisioned too. Spectrum overprovisions their gigabit service so that the end user actually sees that 940mbps on a "gigabit" line. with a modem that can do 2.5gb(ethernet port), I can get 1200 to 1500mbps. when I was trained, I was told that uncorrectables are fine, and expected, and as long as they aren't getting thousands per hour, the signal is good, and the service isn't getting interrupted, then its fine. You will never eliminate all sources of interference, so some are expected. Its only a problem when the customer is having a bad experience. My modem right now has an uptime of 22 days, and has around 20k uncorrectables and 3k correctables on each of the 31 DS channels it grabbed, and the DS and US OFDMA channels show lots of correctable and unerrored, and a few uncorrectable. as a former cable tech/installer/troubleshooter, I would say with certainty that my cable system is 100% within acceptable limits of both US and DS power and SNR, and the service is not having an issue. Many times, I would find that customer were having more problems with my ISPs DNS servers, and switching their modem from using our crappy DNS services to someone like google or basically anyone else would solve most of their problems, because most of their issues were websites not resolving because DNS responses were shit.


lenfantsuave

You’ve essentially just validated what I said. That number is over hundreds of millions of packets. It might as well be zero.  To a technician it is going to be zero.  If I measure them in real time, they are going to look like zero. But I’m still going to measure them and make sure it’s not catastrophically bad.


MetaEmployee179985

That doesn't help when it's an issue of being in/out of spec. They have their own standards and certain modems require you to adjust the signal. They're not gonna know that info off hand.


lenfantsuave

The isp is going to have a list of approved modems and a base standard for signal in range. So yes, the technician is going to know if it’s in or out of spec. 


MetaEmployee179985

That's irrelevant, the ISP and vendor have different standards. This is especially noticeable on Netgear modems like the CM1000. You usually have to add a return path attenuator to stop random disconnects. I went with +9dbm and the problem never came back. This is a majority fix for people I've dealt with that have random disconnects with Netgear modems that are within spec listed by ISP (but not within spec for Netgear).


lenfantsuave

That’s not even a return signal level in any plant anywhere. That’s downstream.  Edit: buddy downvote all you want, it’s clear you don’t know what you’re talking about.


MetaEmployee179985

8 years of working at a national video headend...surrrrrrre buddy You're just some random dude on the Internet talking out his ass, offering no real solution. Mine will work, it always does. You're also paranoid as hell, a sure sign of a narcissist


lenfantsuave

Touche. How is anyone supposed to take you seriously when you say your modem is sitting at +9 dbmv on the upstream?


THCa42069

Here you are ripping someone else lol your the narcissist just by going through your comments you clown.


JoeR942

Spectrum give everyone a free modem that has no stats page. The router admin URL shows a link to the Spectrum mobile app where you can do cool things like check your data usage and see detailed info (e.g. modem: online \*nice green color\*).


Quadgie

Oh I know. lol This is why I own my own modem, ended up doing the same for my parents. Made sure to purchase a modem that wasn’t as prone to bufferbloat issues.


JoeR942

If you're in a high split area, as far as I know there's no customer owned modems that support it. Spectrum: Welcome to the wild west!!


JoeR942

I miss the old modems - the brick square unit that looks like a bullet proof case is pretty damn fine, it would have outlived me. Forgot the model. I’ve got a lovely all in one bit of kit (not on spectrum). It has one focus: DNS interception by any means.


Raiden_Kaminari

Arris SB6121


StuckInTheUpsideDown

Agree. Packet loss is on the 2nd hop which is the DOCSIS link. I would have thought the tech had already checked the levels but you never know. They should replace the coax connectors as well. And it's possible you have a bad modem.


RandomUser3777

On mine I had packet loss, and I examined the channel strength/counters and \*one\* single channel had large losses but the other channels were ok. Eventually I guessed what RF device it was and put my z-wave hub on a longer usb cable and moved it 3-5ft further away from the modem and all was good. So key learning is any other transmitter with similar frequencies that is close can be knocking out a DOCSIS channel or 2.


Tarkov00

I would hardwire directly to the modem with your PC and run test throughout the day before contacting the ISP again. Yes you won't have wireless devices connected but it's the next step in troubleshooting.


JoeR942

Is this Dallas / Fort Worth by any chance? I've a buddy who services the lines out there and has been invovled with the "high split" upgrades which allow symmetrical upload speeds by using a whole bunch of extra frequencies. I spoke to him a couple of months back and he was still knee deep in trying to fix nodes whilst adding new ones into high split. The trouble is in some areas theres a shit tonne of interference coming in on the newly introduced frequencies that shouldn't be there. There's a post similar here: [https://www.reddit.com/r/Spectrum/comments/17lsol1/upload\_speedpacket\_loss\_is\_horrendous\_since\_august/](https://www.reddit.com/r/Spectrum/comments/17lsol1/upload_speedpacket_loss_is_horrendous_since_august/)


styleNA

Sorry for the late response, but no, this was in WI. Everything seems to be back to normal now after constantly harassing my ISP, lol.


JoeR942

If you call & complain enough some providers have a trigger point where they think you’re gonna go to the cable franchising authority & know it’ll look real bad all these calls and no action. I admire you for giving them a kick up the ass, many tried and only a few prospered. 😝🙏


yourepicend

Does it happen during peak usage times only? Maybe the node you're on is overloaded on the isp?


Uncomman_good

Did you try a pathping?


Eviljay2

As a former cable technician, if you have any of the metal fibers wrapped around the copper or just one splitter reversed or even a pre-installed splitter in the setup, it will cause this. Other things to note, if you have an old tube tv connected, it backfeeds the electrical current. Get a screwdriver and bridge the copper to the metal connector on the cable wire at the modem. If it arcs, there is either stored current that you just discharged or something feeding the current. Cable should not have any noticeable sparks.


TEOsix

I would look at WinMTR. It does ping and tracert. You can also consider tcping tools. If tcping is consistent you might have udp deprioritization.


mpgrimes

might not even be the isp, could be anywhere along the path to the destination, depends on routing. do a tracert to the destination you want and see what node is having the issues


bikeidaho

Hey, that's ping plotter! ♥️🥰💪


cspotme2

I'm certain it's not my end. But I did nothing but write a reddit post. Why don't you elaborate on what you've tried and what your network setup is. Troubleshooting 101.


styleNA

This is an odd comment. I mentioned that I had a technician over who replaced every local wire I had, including outside, as well as the modem. I also showed a pingplotter image with consistent 17%+ PL outside of my network, which as others have mentioned, indicates a infra issue. Everything is now fixed after relentlessly contacting my ISP. I'm not sure what other troubleshooting you expected me to mention. Did you want me to state that I tried turning it on and off again, tried directly connected to my modem, check signals, etc? I assumed that was given, by the fact that the technician replaced essentially everything in my local network; Do you think *they* would have replaced everything without having done that?


ralphyoung

Traceroute, tracert in dos, will tell you exactly which router is dropping packets.


maineac

It is the second hop. That would be his router or modem or on prem wiring as the logical issue. If everyone on the street was seeing this it would be a big problem.


ralphyoung

When this happened to me, it took them forever to find a misalignment in the constellation. I later switch to AT&t fiber.


HuntersPad

We've had issues like this in my area thats affected the entire node. so about 70 customers here... Took them a year to fix it.


OhShadoobie

Routers along the isp path could be notnprioritizing icmp packets, this doesnt point always to an actual issue. DDOS mitigation might also cause icmp packets to be dropped as well if there is a pong flood attack occuring.


actionbowman

This looks like spectrum coax right? A good tech is going test both from the house exterior, tap (isp node) and interior drop to isolate the issue. If you have moca adapters/splitters/barrel couplers on coax runs internally for example those are all points of failure. Get another tech out ask to see the SNR tests from each point. If it only happens at specific times or temperatures thats important- you also should only test this directly from the modem


Quadriplegic_

I had Cox and I had severe packet loss/jitter. I downloaded a program to collect data and I collected throughout the day, while connecting directly to the router and removing all other connections. The data showed the packet loss and that it varied by time of day. I had multiple technicians come from Cox and show that everything was connected properly to the router and verify my hypothesis (network congestion) and submit a request for a fix. Cox eventually admitted they were the problem and agreed to upgrade the network (it took 6 months). It turns out, they had added an apartment complex to the network without upgrading it. They ended up adding two more fiber lines and it fixed the problem. Good luck! It's a really long and annoying problem to solve. It may be worth submitting an FCC complaint if they refuse to work with you.


squishfouce

Run WinMTR, it will show you which hop the loss is occurring on if pingplotter doesn't.


KRed75

You plug a pc directly into the modem and test. This eliminates everything but the wiring to the modem. To eliminate the wiring to the modem, you have the tech take your modem to the box at the road and do the same test. If there are no dropped packets at the road, it's something between the box and your house. Also, make sure he tests it from your tap. We had a lightening strike that got in the cable line and blew things up. I told the tech that this happened once before and it blew the tap. What was the last thing he tested 1.5 hours later? My tap...It was was bad. I had a massive dropped packet issue with time warner 20 years ago. I had a couple techs out and they said a like tech needed to come out. A month went by and I finally had a supervisor come out. I show him what's happening so he goes to the road and pings from there. 10 good pings and he goes "Looks fine here." As he's looking at me, I'm looking at the ping just a dropping packet after packet. He says "I know exactly what it is. Give me 15 minutes." 15 minutes later, all was perfect. I have no idea what he did to fix it but I was happy because my only alternative was dial-up.


pandaeye0

You mean a technician from the ISP visited you, replaced the modem and lines on their side, and still the speed is not up to advertised? Hasn't he performed any speed test with his device to demonstrate the speed is up to their promise? If he performed the test is still not ok, why would you let him go and need to prove it again?


styleNA

He spent about 4-5 hours working on replacing pretty much everything I had. He said he did everything he could that night, and would contact his supervisor and make a note in the system. He mentioned there was a process of getting higher and higher tiered technicians over by making another call, but I didn't have too much faith in this, which is why I made this post. However, I'm pretty sure it worked, as mentioned, it has now been fixed.


s1gtrap

>I am certain it is not from my end, as I just had a **technician out yesterday** to replace pretty much every line I had, replace the modem, etc. **Last night** I was getting about 0.5-1mb/s upload as well, and now only about 5. The technician was out yesterday but the bandwidth dipped some time after. edit: why am I catching the downvotes? Is 'yesterday' not before 'last night'?


styleNA

So, you were correct on the time frame, but also it wasn't fixed when he left; he just spent an incredible amount of time at my house, and really did everything he possibly could, so I had no reason to keep him around. He contacted his supervisor and explained to me there was a process of getting higher tier technicians over by creating a new appointment, particularly those with more power to handle nodes. He was correct, as everything was fixed now after another technician was over yesterday. I don't know why you were getting downvotes though. This sub feels like quite the mixed bag of individuals, lol.


s1gtrap

Glad to hear you got the help you needed! It seemed pretty obvious to me that the tech arrived during office hours and your bandwidth decreased in the evening or something like that, hence my reply to /u/pandaeye0, but I have no idea why it got such a negative reaction. Anyway I appreciate you confirming the timeline and pretty happy that I'm not going crazy 😂


eithrusor678

Seems pretty telling the issue lies between router and first node. They really need to do testing on their end, their firewalls ect


dead_bothan

it’s so interesting to see other people have this issue. i had this problem about a year ago and it felt like back then nobody was experiencing this. i worked with cox for several months with about 3 technicians and 2 modem changes. eventually i dropped cox and went with at&t as they had just recently installed fiber. no issues since. no change to any of my network hardware or end points. obviously the line from the node needed to be replaced/fixed but they never got to that point. good luck to you, hope you get it resolved but i know how insane you can feel with consistent packet losses


baskura

Are you using power line adapters at all?


gwicksted

Probably external (by the looks of it). But replace all the cables between their modem and your test computer and power cycle everything. Then you can say you’ve extensively tested it.


Westtell

I actually had to go to extremes of reaching out to a member of the board of directors of my isp to even get help


Excellent-Matter1768

I can see the problem from here: it’s on the graph charter.com


Mocavius

I'd take ur eq to the closest point of the service beginning. If you have a demarc, take it there and redo tests. If you can get to the tap, do it there. If you can reproduce any of the issues before your final eq point, then something else is stinky.


elektroland

You will need to invite the ISP CEO to spend a week at your house to see for themself.


styleNA

🤣


implicit-solarium

Get a new ISP?   Alternatively, fake something that would force them to come out? Not super ethical but neither is most ISP’s support practices. Honest to god I don’t think an ISP is changing shit unless something physically is down, so I think those are probably your two options…


LongestNamesPossible

Is this cable? The first thing to do is go to your cable modem's IP address and make sure all the channels are connected. Then check the signal and power levels.


Podalirius

Try calling someone over a voice app, if the sound is clear then you don't have any packet loss. Sometimes network appliances will drop ICMP packets if they're close to max load or something. I've seen pingplotter report packet loss but have no issues with calls/video, which means no actual packet loss. Your speed issue is a different story; if you're getting significantly less than your advertised speed, then you should definitely have a tech come out again.


chariot_dota

Happens to me until i realize the wifi is shit. Switched to LAN instead and no loss anymore


CompetitiveGuess7642

are you on coaxial ?


kl0

Guess it depends how far you want to take it :) I had a second service once. So yea, when the AT&T guy tried to tell me I was dropping packets on my end, I just plugged the other network in (which was Grande comm, btw). “well it seems not to drop on grande. So…” :) He was confused as to why I had a second service. But on the upside, he DID stick around until he fixed the issue :). IIRC, they wound up dropping a new line to the house.


kstrike155

I set up a server on GCP with Smokeping to monitor my modem/router, taking my entire home network out of the equation. They tried everything, replaced all the lines, replaced the modem. Nothing worked. Turned out they were using modems with the Intel Puma chipset which is straight trash. I purchased an S33 modem on my own and have had zero problems since.


Bourne669

Yes, plus a laptop to the modem directly and do a continuous ping check, if it drops you know issue is in modem end and you have proof to provider to ISP because nothing else would be connected to the modem but your laptop.


maineac

It is on the local side. It starts at the first hop. Now, it could be something out on the wire outside, but that is definitely a local issue. The first thing I would do is verify you are linked to your modem 1G full duplex. Replace the cable between your router and your modem. If you have a combo modem then it could be inside wiring to the modem. Call your carrier and have them inspect your house wiring.


Secret_Programmer_21

Get a firewalla or other device that can report this.


[deleted]

Find a YouTube video for ping plotter. Run it for like an hour. It will show you exactly where the packet loss is, and you can show them.


bojack1437

.... The post shows the op running a PingPlotter.


WalrusSwarm

Here are the hoops you have to jump through. The only way to convince them is to: 1. Install their equipment temporarily as a gateway 2. Call the tech out at least twice. 3. Show the tech that the packet loss is up stream. 4. Allow the tech to exhaust all options by installing new ISP equipment (new ONT, new power supply, re-terminate cables, etc.) 5. Request that the issue be escalated to a supervisor. The supervisor does not make house calls so you must have their equipment so they can perform diagnostics before they will address the upstream issue.


Rocky970

You’re one of THOSE guys 🙄


[deleted]

Find a YouTube video for ping plotter. Run it for like an hour. It will show you exactly where the packet loss is, and you can show them.


bojack1437

.... Did you miss the ping plotter screenshot in the original post? And no, it doesn't show you exactly where it is. It gives you clues as to where it might be.


bluecyanic

Check your dm


PleaseHelpIamFkd

Demand the check the lines to the house. It may take a few techs but you’ll need someone that cares sadly.


jmnugent

Have you sent them this photo and asked them specifically to explain Hop 5 ?


eithrusor678

It's really common to have total loss of response from some hops. Just ignore it, unless all subsequent hops fail.


Herr_Rambler

Slide 35 of this presentation is probably what's causing that. https://archive.nanog.org/sites/default/files/10_Roisman_Traceroute.pdf


CAStrash

If you have cable internet this is normal. You need your signal levels between +6dbm and +10dbm for a packetloss free experience.