Tips for Cisco Wireless Performance on Mobility Express

Recently I was working on my lab network, and I have an 1831 access point and a 3702 AP.   My comments are specific to Mobility Express, which to be fair is just a regular WLC, running on an access point,  with all AP’s in FlexConnect mode only.   The AP’s are responsible for all packet processing, NBAR/AVC, anything you are doing goes on in the AP’s.

Naturally, I wanted to get the most out of my network, but I ran into a few challenges, and I will document them here….

First, I am far from a CLI expert on the WLC stuff,  I have spent most of my life running WLCs with the GUI – but the Mobility Express series GUI is very simple.     I got much better at the CLI during this.

The latest GUI on 8.5 has an “expert” mode now that lets you play with some of the RF settings, the 8.3 version is pretty simplistic.   So I popped in the 8.5.103 version, and was liking the new GUI.     Everything seemed like it was working….     I applaud Cisco for improving the Mobility Express GUI – it was more simple than some home Linksys offerings in the beginning, this is a step in the right direction.

Let me outline my environment….

  1.  I live in a rural area – there is ZERO wireless noise here, and I control the spectrum pretty well.  I don’t deploy stuff without considering the impact.
  2. I have about a dozen client devices
  3. For all my testing – I kicked everyone off 5ghz, and ran on just a single AP.   Nothing else was in the air – I confirmed this using a spectrum analyzer.

Running the latest bit me

Until I had a problem with my Macbook Air (Early 2014 Model).  If you go and look, many people complain about Apple Macbook Air’s and wireless issues – so many different opinions, some blame Apple, some say replace your “router” or access point but I couldn’t find any kind of real problem.

Not a surprise.  I ran 8.5.103 – and I was having weird problems.    All of my clients were fine except my Macbook Air – as long as it was on 2.4ghz, it was fine – but bump it up to 5ghz, and as soon as traffic started flowing – the AP would simply start ignoring the client.  Client thought it was associated, AP saw it as associated — but no traffic moved.     It would sometimes come back, sometimes not, if I bounced client adapter – it would come right back.  2.4 was solid.

Doing what I always tell my clients – run the “Gold Star” release in this case 8.3.122 – So I put that version in, and let the APs upgrade.    Everything seemed better now – connectivity was solid.      After my findings below, I went back to test 8.5.103 again…

AVC Hurting Performance

So being that it was “working”  I switched to performance testing.   I run a iPERF3 server on my QNAP here at home – confirming performance I was getting 995mb/sec from my wired desktop to the NAS…  Ok we are good to test.

My Macbook Air was connected with the following…

Performance Signal Strength: -53 dBm

Signal Quality: 43 dB

Connection Speed: 867 Mbps

Channel Width: 80 MHz

Capabilities 802.11ac (5GHz) Spatial Stream: 2

Time for a test…

Connecting to host QNAP, port 5201
[ 4] local port 56551 connected to QNAP port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 15.5 MBytes 130 Mbits/sec
[ 4] 1.00-2.00 sec 16.1 MBytes 135 Mbits/sec
[ 4] 2.00-3.00 sec 16.0 MBytes 134 Mbits/sec
[ 4] 3.00-4.00 sec 15.9 MBytes 133 Mbits/sec
[ 4] 4.00-5.00 sec 16.1 MBytes 135 Mbits/sec
[ 4] 5.00-6.00 sec 15.8 MBytes 133 Mbits/sec
[ 4] 6.00-7.00 sec 15.8 MBytes 132 Mbits/sec

Ok this isn’t right…  Something isn’t working….   So I contacted my good friend @wifijanitor – Steve to bounce some ideas off him.    We quickly got to “It’s all configured correctly”
So I started disabling this and that 802.11(insert feature here) and everything one by one. Problem remained.
Finally, I disabled AVC – Application Visibility and Control…

[ 4] 71.00-72.00 sec 46.4 MBytes 390 Mbits/sec

[ 4] 72.00-73.00 sec 46.3 MBytes 389 Mbits/sec

Well look at this…    The only thing we could figure out is that the AP must be getting hammered by the AVC…   So, I investigated that….

AP CPU with AVC Enabled

Whoa, that is 100%…  This is with my iPerf, i’m getting 140-150 mbit. Ok, let’s try with it disabled.
Screen Shot 2017-09-13 at 9.46.09 PM

AP CPU with AVC Disabled – Heavy Load with iPerf

Whoa…  That’s not a good thing…    That means even the performance i’m getting now is probably being hampered by the CPU on board…   Close to 400mbit throughput, and the CPU is high.  According to the system it is nothing but packet process.  There has to be  a choke point…    I wonder what would happen if I had more CPU – i’m not able to clear up any more CPU, everything (I think) is disabled.
Screen Shot 2017-09-13 at 9.46.09 PM

AP CPU with AVC Disabled – 100 Mbit Stream

Ok so i’m trying to prove my theory…  This is AVC Disabled, 100MB Stream using iPerf.  About 30% CPU utilization…

Screen Shot 2017-09-13 at 10.17.18 PM

AP CPU with AVC Enabled – 100 Mbit Stream

Now I re-enable AVC and run the exact same 100mbit stream.   wow ok we are looking at 75%-ish cpu.    Clearly AVC is causing a CPU bump – that has to be my problem at higher speeds.

Screen Shot 2017-09-13 at 10.12.25 PM

Conclusions and Recommendations

– With AVC running in FlexConnect mode, the AP is responsible for the nBAR engine, which is limited compared to what you get in a real WLC.    If you need/want AVC – plan on installing a full WLC, between the limited AVC capability (well document) in FlexConnect mode, and the un forseen performance issues I have seen (not well documented)  It shouldn’t be used in Mobility Express or FlexConnect installs.
– Running the latest code can bite you (I knew this!)
– Always validate your installations, not just for connectivity, but for performance
– If you are using Mobility Express – Learn the CLi, because there are just some things you cannot do in the GUI.
– I did go back to the latest 8.5 release to see if AVC was the cause of my 5GHZ issues in 8.5.103 – but it was not.
Advertisements

Cisco Connect Toronto with Chuck Robbins

In a splash we are not used to seeing it has been announced that at Cisco Connect Toronto on October 12 (Also known as “Mini Live”) – the keynote speaker will be Chuck Robbins.

chuck_robbins_cisco_ceo.0

In the past the Cisco Canada President – currently  Rola Dagher would keynote this event, but this year with Chuck Robbins coming on site, I think we will see an increase in attendees.

But will Chuck Robbins upstage Rola Dagher?   Will they stand side by side on stage?   Will Chuck and Rola have sit down and talk about technology and specifics about Canada?

This could be a pivotal moment in history for Cisco Canada – as in the past, the great white north has felt cut off from the super power that is Cisco USA.   With Chuck coming here, it may give customers a feeling of connection back to the mother-ship in California.

No doubt the focus of conversation from Chuck Robbins will be DNA-Centre, SD-Access and Catalyst 9K technologies – but how will he customize the keynote for the Canadian audience.

I will be there – and will bring reaction back to the blog.

Information on the event CLICK HERE

 

Meraki Launches Community

In order to enable collaboration between customers, the team at Cisco / Meraki has launched a new community page located at

community.meraki.com

MerakiCommunity

The program is aimed at partners, customers, and technical folks.    As you know the Meraki team has always been about “What do you want to see”.   Consider the success of the “Make a wish program”

This is a great place to discuss problems, ideas and solutions – and Meraki will be watching.   This is a great place where “Make A Wish” could grow into real discussion by the Meraki community to help push ideas back directly into the Meraki team.

Sign up today, and join the discussion.

Meraki changes cloud IP’s

 

Some customers have very stringent outbound firewall rules (Oh, and good on you by the way!) – just an FYI, Meraki is about to change the IP’s of their back end gear on some of their shards.

In an email to customers blasted out from the green heavens today, Cisco/Meraki let customers know that they are going to make some changes in the back end with different control IP addresses.

The good news is that if you forget, or don’t make the change your network will not go down, but you won’t be able to make any changes to configuration, and use data will be cached.

So, go ahead and make that change now before you lose connectivity.     This comes after Meraki had some block storage issues a few weeks ago which saw some configuration data impacted.   This may be part of the remediation and resiliency upgrades to deal with that situation, but I don’t know and cannot confirm (Looking into it).

 

MerakiLogo

Dear Meraki Customer,

As part of ongoing efforts to improve the performance and resiliency of the Meraki Cloud we will be changing the IP addresses used by Cisco Meraki devices to contact the Meraki Cloud.

In order to ensure that customers have time to make these updates, the change will take place 8 weeks from first notice, or after all affected networks have updated their firewalls, whichever comes first. You can prepare for this change by opening up access in your firewall to the IPs and ports listed on your organisation’s Firewall Information page ( https://dashboard.meraki.com/manage/support/firewall_configuration ).

Your Meraki network will continue to operate, but your Meraki devices may experience degraded performance and connectivity to the Meraki cloud if your firewall rules are not modified to include the IPs and ports listed on that page.

 

If you have any questions regarding this message, please contact Meraki Support at support@meraki.com or +1-415-632-5994.

Cisco releases Spark and Jabber Interop

Good news the Cisco Spark and Cisco Jabber interoperability is up and running and working.   People using Jabber 11.9+ can now see and chat with people on Cisco Spark, but one to one only.   Soon spark spaces will also be supported.  There is some presence sharing between the platforms.

jabberplusspark

The Jabber Platform does not support access to Spark Spaces, ad-hoc group chats, the share of attachments or screen sharing across platforms at this time.  So 1 to 1 chat is all users get for now.   Details of new features in this link.

A few deployment requirements….

Jabber must be CLOUD DEPLOYED – so no on premise deployments are currently supported.  You will need to contact Cisco to get your system added to the Cisco Spark platform identity service.

This is a long time coming feature many clients have been asking for as customer embrace both platforms, however, most clients looking to migrate are probably using an on site implementation of Jabber today.  Hopefully, Cisco will support on premise migration strategies soon.

The FlightChops teams reaches 100K Subs!

So many of you know I am good friends with Steve Thorne from www.flightchops.com and also his amazing YouTube channel located HERE

I want to take this moment to congratulate Steve and his team for reaching 100K subs, as of this writing he is actually at 107K.   This has been an amazing story about someone who was passionate about a topic, took that online in their own way.   People attacked Steve alot in the beginning, and even now for posting mistakes, troubles and pitfalls of learning to fly but the truth is – that is what made his channel popular.

Sponsors have taken note, and big names too like Bose and ForeFlight have put some support behind Steve.  Even with the big name support he still receives a significant contribution from Patreon (including me) and he never forgets those who got him here, by running many contests for everything from Bose Aviation headsets to a San Juan Islands Adventure trip!

Content is king – bottom line – and Steve and his team of editors and videographers have amassed a ton of content and gone from twice monthly to sometimes 4 times per month publishing this amazing content.      Steve continues to do this his way, and sponsors do not impose on content.   His “day job” of video and media production has brought a significant professional flair and production quality to his episodes and that production quality has been steadily increasing.

Will this ever reach “Mainstream” television?  You mean YouTube isn’t mainstream?   I would appear to me that the likes of “Outdoor Network” are only interested in fake shows auctioning off storage units sadly.

So if you are a private pilot, or just an airplane nerd like me – go and check out his humble little channel and I promise you will learn something along the way, as Steve says “Keep your Flight Chops Sharp!”

Congrats again Steve on your channels success!