This is my recollection of Course II, Week 6 of the Google Professional IT Certification Course on coursera.org.
. . .
Troubleshooting and the Future of Networking
Because computer networking is so complicated and there are so many different protocols working together (sometimes) that problems are common, and troubleshooting is essential.
Error detection is the ability of a protocol or program to determine that a problem exists. Error recovery is the ability of a program or protocol to attempt to fix the problem.
We’re going to go over the most common techniques and tools used to troubleshoot network issues.
This could be some useful stuff. We are going to go over MacOS, Windows and Linux networking troubleshooting techniques.
At the end of this week we are going to cover the future of networking with the cloud and IPv6.
Ping: Internet Control Message Protocol
The most common networking problem is, completely obviously, a failure of something establishing a connection to something else. Network errors are very common and troubleshooting them is one of the most important skills to have.
When there is a network error the device that detects it must have some way to tell the source that there is a problem.
Maybe a router can’t reach a destination, or a port is unavailable. Whatever it may be, internet control message protocol (ICMP) is used to communicate these messages. ICMP is usually used by a router or remote host to notify the original sender that there has been a problem.
An ICMP packet has a header with a few fields and a data section that is used by a host to determine which transmissions created the error.
- Type field: 8 bits, specifies what type of message is being delivered, such as “destination unreachable” or “time exceeded.”
- Code field: 8 bits, this is a specific detail about the type of message, such as a code indicating “destination port”
- Checksum: 16 bits,
- Rest of Header (Yes, that is what it is called): 32 bits, an optional field used by some types and codes to send more data about the error.
- Data payload: tells the recipient of the ICMP packet which transmission caused the error being reported. It contains the entire IP header and the first 8 bytes of the data payload of the offending packet.
ICMP was not developed for humans to read or interact with. Instead, it allows machines to communicate with each other automatically. There is, however, a tool and two message types that are very useful to humans.
Ping is a tool that exists on almost every operating system. Ping is very simple and is basically the same in every operating system. Ping allows you to send an ICMP message called echo request, which asks the destination if it is there. If it is up on the network and working properly, it will respond with another ICMP message called echo reply.
Ping can be accessed from the command line in almost any operating system, usually by typing the command “ping” and adding an IP address or FQDN:
The output of the ping command will also almost look the same in most operating systems. It will include the time it took for the roundtrip to the destination and back:
Reply from 126.96.36.199 bytes=32 time=6ms TTL:56
Reply from 188.8.131.52 bytes=32 time=5ms TTL:56
Reply from 184.108.40.206 bytes=32 time=4ms TTL:56
Reply from 220.127.116.11 bytes=32 time=4ms TTL:56
On Linux and MacOS ping will run until interrupted, usually by pressing <ctrl c>. In Windows, ping sends four echo requests. Ping can also be modified to change the size of the message, how many requests to send, etc., depending on the operating system.
Ping tells you if you can reach a device, and provides some level of connection quality information. But Network troubleshooting will often involve problems several devices away.
Traceroute “lets you discover the path between two nodes, and gives you information about each hop along the way.” Traceroute works by employing the TTL (time to live) field in the IP layer. Remember, each router decrements the TTL field by 1 each time it forwards a packet, and when TTL=0 an ICMP “time exceeded” message is sent to the originating host.
Traceroute works by cleverly assigning 1 to the TTL field of the first packet it sends, so that it will be discarded at the first router hop. The second packet is assigned 2, so that it is discarded at the second hop, and so on until the host finally sends a packet to the intended destination, and the ICMP Time Exceeded messages stop coming back. Traceroute sends three identical packets to each hop.
Traceroute outputs some simple information, which may vary slightly from OS to OS. Each line will show the number of the hop and the roundtrip time for each of the three packets, the IP address of the device at that hop, and a hostname if tracert can resolve one.
1 10.0.0.1 (hostname, if available) 2.34ms 2.21ms 2.21ms
In Linux and MacOS traceroute sends UDP packets to high port numbers, and in Windows (the command is shortened to “tracert”) and by default uses ICMP echo requests.
Two additional tools to use are mtr in Linus/MacOS, and pathping in Windows. These act as long-term traceroute. Mtr will update data about the traceroute in realtime. Pathping runs for 50 seconds and displays the data after that. (I just ran the pathping command in PowerShell and it says it is computing for 225 seconds. Maybe this has changed from cmd to poSh.)
Testing Port Connectivity
While there are many ways to test connectivity at the network layer, testing at the transport layer is also important. Here are two “super powerful” tools you can use:
Netcat on Linux and MacOS and Test-NetConnection on Windows. Netcat is initiated with the command “nc” and has two mandatory arguments, host and port. For example, you could run
user@user-host:~$ nc google.com 80
which is going to try to make a connection to google.com on port 80. If it fails the command will terminate, and if it succeeds it will show a blinking cursor, waiting for more input. Use the “-z” flag to initiate zero input/output mode, which is used when you only want to know the status of the port. Using the “-v” flag creates verbose output, which is better for human readers. Do not use -v for output to scripts.
C:\Users\Adm_Underscorn> Test-NetConnection google.com
Is a similar command in Windows. If you only use the hostname it will default to ICMP echo request, much like ping, only it will display much more data, including the data link layer protocol being used. Use the “-port” flag to test the connectivity to that port.
There is a lot more to these commands, so please enjoy the upcoming additional reading. YAY!
Reading: Testing Port Connectivity
Digging into DNS
Name Resolution Tools
Name resolution is normally handled by software, and regular users should not have to worry about it. IT support may require you to use some tools to manually assess name resolution. Here are some useful ones to know.
- nslookup is available on all three major operating systems. As an example, let’s say you want to know the IP address of com. All you do is enter the command and then the hostname, and it will return the server that was used to perform the request and the resolution result:
C:\users\user: nslookup twitter.com Server: cdns01.comcast.net Address: 2001:558:feed::1 Non-authoritative answer: Name: twitter.com Addresses: 18.104.22.168
Nslookup can also be used in an interactive mode, just by entering only the command “nslookup”. You will see a cursor or bracket, prompting you to enter a series of commands. If you enter “server” and then an address, all the name resolution queries you perform after will be made using that server instead of the default name server.
Entering the command “set type=” followed by a resource record type (default is A-record) such as AAAA (quad-A), MX, or text records.
You can also use the command “set debug” which allows the tool to display “full response packets, including any intermediary requests and all of their contents.” (I am not sure what this means and they do not explain it in any more detail in this video, except that it contains a lot of data.)
- This video began by referencing “tools” and then mentioned one tool, with several functions built into it. This seems a little lame but hey, here we are, already done with the second bullet point…
Public DNS Servers
Your ISP will almost always give you access to a recursive name server as part of your internet service, but most businesses will also run their own DNS services for internal devices. This allows the business to refer to a printer by a name, for example, rather than by an IP address. This requires the business to operate its own DNS server.
Another option that is growing in popularity is the “DNS as a service provider,” which we will go over later.
Whichever of these three systems your network uses, you are going to need to be able to test DNS functionality.
Some internet organizations run what are known as public DNS servers, which provide name resolution for anyone, for free, and can be very useful in troubleshooting.
One of the oldest public DNS services is operated by Level 3 Communications, an ISP so large that they mostly sell their services to other ISPs.
The Level 3 public DNS addresses are:
While this service has been available for almost 20 years, there is something of a mystery surrounding the Level 3 public DNS service. They don’t advertise it, they have never acknowledged that they operate it, and nobody knows why. Weird.
Google also operates a public DNS service, located at:
Google publicly states that their DNS servers are open to everyone and can be used for free.
Most public DNS are globally available through anycast. There are many other public DNS servers available, but these two examples have the benefit of being very easy to remember.
Remember to always do your research before re-configuring DNS on your network. It is quite common for attackers to hijack DNS requests and return bad responses that direct traffic to malicious sites.
Make sure that you are using a reputable public DNS server, and it is considered best practice to always use your ISP’s DNS server outside of troubleshooting scenarios. Also, most public DNS servers respond to echo requests, so it is a good way to check network connections in a general way.
(There is a lot of talk about the new 22.214.171.124 DNS server from Cloudflare, launched, to much skepticism, on April Fool’s Day, 2018.)
DNS Registration and Expiration
Quick refresher, here: DNS is a global system managed in a “tiered hierarchy” with ICANN at the top. The fact that it is global means that domain names need to be unique.
Registrars are organizations that assign domain names to organizations or individuals. Originally, there were only a few registrars, notably, a company called Network Solutions, Inc. They were responsible for most domains that were not country-specific. After the internet grew in popularity, the US government (and Network Solutions Inc.) agreed to allow other companies to sell domain names. There are now hundreds of registrars.
The process is simple—just create an account with a registrar, find an available domain that you want using their website, then select the length of the registration and pay the fee.
You can allow the registrar’s name servers to be the authoritative name server for your domain or you can configure your own.
Domain names can be transferred from one party to another, or from one registrar to another. This is done by using a special string of characters in a text record that confirms ownership by both parties, after which ownership moves to the new party.
It is very important to know when your domain name is going to expire so you can be ready to renew, otherwise someone—anyone—can register your old domain.
Before DNS was established it was apparent that there would need to be a language-based system that operators could use with networked devices. But numbers are the natural way that computers communicate. Obviously!
The original method that allowed numbered network addresses to be represented by and correlate to words was through what were called hosts files.
Hosts files is a “flat” file that contains, line by line, a network address followed by a host name that can be referred to in place of that numbered address. For example: “126.96.36.199 webserver” means that an operator could just refer to “webserver” instead of remembering the numbers. You could use the command “ping webserver” and the target would be the device at 188.8.131.52.
Hosts files are referred to by the networking stack of the operating system, and an entry there will translate to anywhere a network address is used.
Hosts files are “ancient” but they are still in use in all moderns operating systems, including mobile. This is because they are still used for loopback addresses, which are network addresses that allow you to send traffic “to yourself.” This traffic bypasses the entire network, never leaving the node.
On IPv4 the loopback address is 127.0.0.1, and is still configured on every operating system in a hosts file. Almost every hosts file will contain, if nothing else, the line “127.0.0.1 localhost” and is often followed by “::1 localhost”, which is the loopback for IPv6.
While DNS is almost universally adopted, it is important to know that hosts files are still used, especially because they are a common target for computer viruses as a way of disrupting traffic.
One important thing to know about hosts files is that they are always consulted before a DNS resolution request is made. This means a single computer can be forced to always think a domain points to a certain IP address.
>>>goddammit another quiz>>>
What is the Cloud?
The cloud is almost anything anyone talks about anymore. Cloud storage, cloud servers, cloud-based antivirus? What are these things???
The cloud is just a concept, not anything more. Cloud computing is a configuration where resources are “provisioned in a shareable way” so that many users can access them when needed. Cloud computing means that many companies can provide you (and each other) with different services.
Hardware virtualization is an essential concept to cloud computing. Virtualization allows a physical machine (host) to be abstracted into multiple logical machines (guests) running on the same hardware.
Because the operating system needs to communicate with hardware in certain ways, virtualization platforms use what is known as a hypervisor, which is software that manages virtual machines and provides them with access to hardware. This creates a platform that is identical to the “actual” hardware.
One machine can simultaneously run several virtual machines, all with independent operating systems. Cloud computing just expands this principal, giving any number of users access to a range of “computers” operating on a cloud provider’s physical hardware.
To clarify, if a business needs four servers, all with different needs and operating systems, (a Windows mail server, a Linux database, a Linux DNS server, whatever), and the most economical machines they can get has 8GB RAM and x storage, they will have, at a minimum, to physically own 32 GB of RAM. But the DNS server only needs 2, and the mail server only needs 4. The database server needs the most, but only at peak times. With cloud computing, each of those servers can be setup as a virtual machine, on rented hardware. When the database server needs lots more computing power the cloud provider will allocate it and keep it running well. When the DNS server isn’t doing anything it won’t be taking up resources it doesn’t need.
This makes a huge amount of economic sense for many organizations. Add to that if the cloud provider is also offering other services in addition to computing power. Backup solutions, for instance, easily integrate with cloud computing.
These examples are all public clouds. There are, of course, private clouds, used by a single large company and usually hosted by itself. There is also the concept of the hybrid cloud, which integrates public and private cloud systems.
Everything as a Service
This is the idea that X can be a service provided by a cloud-based platform. X-as-a-service, or XaaS. We just went over some examples of infrastructure as a service, or IaaS, which means that you pay someone for server and storage service. There is also quite commonly now Platform, or PaaS, and software, or SaaS.
Web developers don’t need to build a server to test their new web app, they can just buy some time and space in a virtual environment from a PaaS provider.
Gmail is a very popular example of SaaS – you don’t need to download, install, and then run the Google email program, you just access it from a browser. This is due to the advancement of browsers and features that can be added and built in. More and more businesses and organizations are turning to SaaS solutions.
Cloud storage simply means that a customer pays a provider to keep their data secure and accessible. Because hard drives experience failures at a regular rate, they are a weak point for any data storage system.
Many cloud storage providers have a global presence, making it possible to store copies of data in distant locations. Cloud storage can be available based on how much space you need, whereas with a hard drive at your location you will necessarily have some unused space that you have paid for.
>>>discussion “Digital Footprints in the Cloud”>>>
This was just a discussion prompt about what it is like to use cloud services in your day-to-day life. There were two anwers I saw from most students, either “I love it it’s so easy,” and “the government can read my gmail.”
IPv6 Addressing and Subnetting
We are all out of IP addresses. 32-bit IPv4 address space is all gone, so we now have IPv6 being implemented. (What happened to IPv5? It was an experimental protocol that attempted to deal with connection states or something, and was never widely adopted.)
IPv4, as you will recall, uses a 32-bit address which means there are about 4.2 billion addresses. IPv6, however, is a 128-bit address, creating 2^128 unique addresses. That is a lot. That may be about as many atoms exist in the universe, so we probably won’t run out.
IPv4 is written in 4 octets. IPv6 is written as 8 groups of 16 bits each, like this:
Any IPv6 address that begins with “2001:0db8” is reserved for education and documentation purposes, such as online course material. That means there are 18 quintillion addresses just for education! Who needs that much educations???
There are two rules for abbreviating IPv6 addresses. The first is to remove any leading zeroes from a group, and the second is that any consecutive groups composed of only zeroes can be replaced by two colons. That means our above address could be shortened using the first method:
And now with the second method:
Remember way back to twenty minutes ago when we went over the IPv4 loopback address of 127.0.0.1? The IPv6 version is
There are many other reserved address spaces in IPv6.
FF00:: s reserved for multicast, which is a way of addressing groups of hosts all at once.
FE80:: is used for link local unicast, which “allow for local network segment communications” and are configured by using a host MAC address. This allows hosts to receive network configurations, similar to how DHCP works.
Unlike with IPv4, there is no need to worry about reserving address space. With an IPv6 address, the first 64 bits are the Network ID, and the second 64 bits are the host ID.
Even though there is more than enough address space on an IPv6 network, subnetting is still used for administrative purposes, and uses the same CIDR notation that we are already experts at understanding and clearly, calmly explaining to others. The same method is used to define a subnet mask against the network ID.
An IPv6 header has some improvements made upon the old IPv4 header. The header is made of the following fields:
- Version Field: 4 bits; which version of IP is in use. Just like the IPv4 header. Makes sense!
- Traffic Class Field: 8 bits; defines the kind of traffic inside the IP datagram, and can be used to prioritize different kinds of traffic.
- Flow Label Field: 20 bits; used along with traffic class field to allow routers to make decisions about QoS for that traffic.
- Payload Length Field: 16 bits; defines the length of the datagram’s data payload section
- Next Header Field: defines which of the optional header fields directly follows this current one. IPv6 contains a lot more data overhead, beginning with sheer length of addresses. The next header field allows optional headers to be skipped if they are unused, or configured in a chain if there are many being used.
- Hop Limit Field: 8 bits; Just like the TTL field in a IPv4 header.
- Source Address: 128 bits
- Destination Address: 128 bits
- This is where the “next header” would come if one was being sent; If not,:
- Data Payload Field: Length as specified.
IPv6 and IPv4 Harmony
It is essential to make IPv4 and IPv6 work well together. Many devices still cannot “speak” IPv6, but the two systems still need to communicate with each other.
IPv4 Mapped Address Space allows direct correlation of an IPv4 address to a specially set-aside IPv6 address. For example, 192.168.1.1 = 0:0:0:0:0:ffff:d1ad:35a7
If an IPv6 address begins with 80 zeroes and then 16 ones, the following 32 bits are recognized to be part of the IPv4 mapped address space, and the last 32 bits are the same as the IPv4 address.
This allows individual networks to be updated to IPv6 while still allowing traffic to pass through the older IPv4 infrastructure of the internet backbone.
IPv6 Tunnels are created by placing an IPv6 server at either end of a connection and taking incoming IPv6 traffic and encapsulating it in regular IPv4 datagrams. This is then sent over the IPv4 network to another IPv6 tunnel server, where it is de-encapsulated and sent on to the next IPv6 network.
There are now IPv6 Tunnel brokers, who handle this traffic for your network, so that you don’t have to add equipment. There are many new and competing protocols for this, which are still being worked out. Someday, IPv6 will be adopted across the whole internet, and we will no longer need these tunnels.
IPv6 and IPv4 Harmony (reading)
Some reading on competing IPv6 tunnel protocols.
>>>2 graded quizzes>>>
One of the “quizzes” is actually an exercise where you have to condense 10 IPv6 addresses. It is very important to remember that you can only condense two consecutive groups of four zeroes once, and if, later in the address, there is another group of four zeroes, you have to delete three of them, not convert them all to a double colon. This is because when a device re-forms that address to the full 128 bits, it will not know how many zeroes were in each place now represented by a double colon. So, to get through the exercise, first I condense the first set of all zero groups to a double colon, then I remove any leading zeroes from their groups, and if a group is four zeroes, it becomes one zero.
That better make sense to me tomorrow.
Course Wrap Up
>>>Discussion “Your Learning Journey”
Course Wrap Up
I’m a great student, this course was complicated, there’s lots of stuff to know about networking, enjoy the next one.
Alex Career Path
This guy again… learn the skills, there’s lots to know but you can do it, blah blah he studied philosophy, okay, thanks.
That’s it for week 12, the final week of Course II.
Man, they really went all-out on the fake-looking certificates, huh?
See you next week for Week 1 of Course III. (Week 13 in my running total.)