BLUF: No “Smoking Gun” Reason Found – Latency Less of a Problem Last Three Days - Troubleshooting Continues.
Background: On Wednesday 18 Feb APAN users began experiencing significant performance latency within the APAN platform. Several events around the globe who were using APAN communicated to our Help Desk that the APAN latency was significantly and negatively impacting their event mission / business objectives. The APAN latency was disrupting operations to a point where we decided to recommend an APAN outage notification be released.
The APAN Application Service Provider Software & SQL Database Engineers along with DISA DECC System Admin and Network Engineers (where APAN Production System resides) have conducted several conference calls to troubleshoot the problem. During our troubleshooting activities, we scoured the system logs, rebooted various APAN components, reviewed operating system metrics, reviewed network monitoring vehicles, and even considered the possibility of network defense issues. We still have not narrowed down what specifically caused the problem. On Friday afternoon Hawaii time, 23 Feb, we decided to disengage various “behind the scenes" services that are routinely scheduled to occur on APAN. This includes various logging events, actions that synchronize Zimbra community with SharePoint, metrics computing routines and various other system operations. This effort appears to have reduced the latency issues. However we still have not discovered the reason why. These are relatively minor services that run on APAN in the background, so we still need to monitor APAN over the next few days while we continue searching for more clues to the latency and why removal of these services had a positive impact on reducing latency.
The APAN Applications Service Provider along with the DECC engineers have plans to implement additional troubleshooting actions if the latency issue raises its ugly head again.
Thank You for your Understanding and Support. We will continue to press.
//S//APAN Applications Service Provider and DECC Egineering Teams
As you are well aware by now APAN has been having significant latency issues over the past week to a point where APAN is unusable. The UISS - APAN DISA Program Management Office, UISS-APAN Applications Service Provider (ASP) and the DISA Defense Enterprise Computing Center (DECC) software / hardware / network engineers are hard at work troubleshooting the problem.
We will use all our assets and options available to us until we find and fix the issue (s).
The monthly meeting of the APAN SharePoint User Group will take place on Weds 11 Feb at 0830 HST (1330 EST/1830 GMT). We will talk about the latest APAN SharePoint News and what's going on with the development team. There will be 2 training items. We will present how to incorporate Easy Tabs 5.0 on your pages so that they have neat tabs instead of just scrolling and scrolling. In addition, we will demonstrate how to add a survey to your site. We will show how you can provide a Thank You screen and redirect your users so that they don't see the input of others in the survey. The meeting will conclude with an open forum for users of all levels. The meeting should last from 40-60 minutes.
The meeting will be held via APAN Connect. Visit the ASPUG Homepage for details.
We have one more server to build and move over to the APAN production environment to complete the tech refresh. Unfortunately that means one more service interruption. We will review all events and come up with a time with the least amount of interruptions.
Unfortunately the tech refresh of the production servers in Alabama did not go as smoothly as we wanted, but then again what ever does. During the month of January we had numerous problems with SharePoint and other APAN applications (map, passport and latency). We have attributed this pretty much to the changes made to the production environment or the changes not made to the production environment. Unfortunately the production environment and the development/testing/replication environments are not in the same location which makes it more difficult to schedule and coordinate. However, APAN seems to be "settling" down (an advanced computer networking term) :) and we can now move on to our next big change, APAN v8.0.
APAN version 8.0 has been a major component of our product teams development for the past eight months. This is a significant software change and therefore plan on implementing a strategic communications program by the end of February. We also plan on having a temporary "beta" site up for some of our power users to get a head start on the changes and provide feedback to our product teams prior to deployment to the production environment. We don't have a specific date for the APAN v8.0 deployment at this time because we want to ensure our customers are ready for it, it works and it does not interfere with the major operational events for your organization. In the meantime we will start to provide regular updates via this blog and by other means concerning APAN v8.0.
Thank you for your patience and understanding. If you have any questions please respond to this blog entry.
Aloha,UISS-APAN Applications Service Provider Team
We have documented your exercise / conference dates provide to us by our APAN Support team and have added it to our APAN support schedule spreadsheet.
As of Friday 16 Jan at 1624 HST, we only require one more additional Authorized Service Interruption to finish up the tech refresh and SharePoint server rebuild. However, the only day we can conduct this last ASI is on Wednesday 21 January from 1000 HST/ 1500 EST until 1400 HST / 200 EST. Regret any inconvenience this may cause. APAN will be “intermittent” during this time period. If all goes well on Wednesday 21 Jan, this should be the final ASI associated to the APAN Tech Refresh.
Thank you for your understanding & continued support.
UISS-APAN Application Service Provider (ASP) Team
We are announcing the kickoff of the APAN SharePoint User Group (ASPUG). The ASPUG will keep users informed about APAN SharePoint activities. The ASPUG will provide a forum for exchanging ideas and information. The ASPUG is not meant to replace the APAN Support Desk or the Customer Engagement Specialists.
The success of ASPUG depends on the participation of the community.
Please join us online at our first meeting on Thursday, 15 Jan at 0830 HST/1330 EST/1830 GMT using our APAN Connect Room. https://connect.apan.org/aspug/?launcher=false (Link is on our web page.) If you cannot attend, the meeting will be recorded and posted. We will include a brief session on how to add a map from map.apan.org to your SharePoint site.
Our web site is up and running at
You must sign in with your APAN account to contribute to the web site.
Please note that there are several planned APAN maintenance periods for this week. The site might not be available during the maintenance.
APAN has received the Support Tickets submitted by customers outlining their required dates / times for uninterrupted APAN operations.
The DISA Unclassified Information Sharing Service (UISS), Defense Enterprise Computing Center (DECC) and APAN Applications Service Provider (ASP) Teams have developed a plan to minimize interruptions to your operations. However in order to execute this plan and complete the APAN tech refresh before it impacts our customers events, APAN outages will need to occur on 12, 13 & 14 January.
The ASI's for next week will be up to six hours each day on 12, 13 & 14 January. A banner on the front page of APAN will identify the time periods.
If additional information is required to be sent to APAN concerning this subject please send to: https://apan.org/pages/supportticket
Thank you for your patience, feedback and support.UISS-APAN Applications Service Provider (ASP) Team(808) 472-7732
Is your organization using APAN SharePoint Site or Group for an important operation or conference in January & February?
APAN is planning its final tech refresh phase of its production environment. The last phase includes the server rebuild & refresh of the SharePoint servers. Please let our APAN Support team know what specific dates (and times if known) are important to your organization to maintain APAN services in January and February 2015. Send your event / exercise dates in an APAN Support Ticket located here: https://apan.org/pages/supportticket
Thanks for your understanding and support.
UISS-APAN Applications Service Provider Team
APAN will be conducting various ASI's between 18 - 31 December in an ongoing project to refresh / replace APAN servers within the Production Environment. APAN access will be intermittent during these time periods and as usual a banner on APAN's home page will identify the date and time of the ASI. Regret any inconvenience this may cause our customers.However, IF there is an operational requirement to keep APAN operational for a specific time period or day during a planned ASI, please call and / or submit an APAN Support Ticket with your information.
Thank You for your Consideration,UISS - APAN Application Service Provider Team
Do you need to brief leadership on APAN capabilities? Do you want to better understand how APAN fits into the Unclassified Information Sharing Services portfolio provided by DISA? Do you need a brochure or handout about APAN for your own event? APAN has existing documentation and briefing material for your use. We encourage you to visit the Support Info media gallery to leverage these existing resources:
Material in this gallery was updated as recently as 1 December 14.
If you need any assistance implementing or customizing the documentation found in the Support Info media gallery, please submit an APAN Support Ticket here: https://community.apan.org/support/p/supportticket.aspx or call the APAN 24 hour Help Desk at (808) 472-3361 or click on the "Support" tab from anywhere in APAN. After submitting your request, an APAN Team Member will reach out to you to provide assistance.
Next week from 1-5 December, the APAN Applications Service Provider Technical Manager, Project Manager, 2 Knowledge Managers, and a Software Developer will be manning one of the CCEM break-out rooms at the Davis Center. If you are attending the CCEM and would like to discuss likes & dislikes about APAN, please stop by the room. We will be presenting a brief on last years and upcoming years software development, road ahead and providing a preview of our new APAN v8.0 which will have some significant changes to APAN group development & ownership. We look forward to meeting you.
Come see our new APAN display on the 2nd floor!
Click here to request access to the CCEM collaboration group on APAN: https://community.apan.org/CCEMpublic/ccemprivate/default.aspx
Below are submitted APAN Authorized Service Interruptions (ASI) request for period 24-26 November 2014:
Monday 24 Nov: Tech Refresh - 0400-1200 HST (241400-242200 UTC) (APAN Intermittent service outages)Tuesday 25 Nov: Patching - 1000-1600 HST (252000-260200 UTC) APAN Intermittent service outages)Wednesday 26 Nov: SharePoint Maint / SQL SP - 0800-1200 HST (261800-262200 UTC) (Intermittent SharePoint only service outages)
APAN has developed and deployed a Chat App for Android OS and iOS devices. The APAN Chat App for Android OS devices can be downloaded from Google Play Store and iOS devices from the Apple App Store. The APAN Chat App allows APAN members to conduct peer-to-peer chat with ANY of their APAN colleagues. Additionally, the APAN Chat App provides the ability for APAN members to use the APAN Translation capability currently available on the full APAN platform.
The APAN community created to support organizations responding to the Ebola outbreak in western Africa, the Ebola Response Network community, is open to the public and can be found here: https://community.apan.org/apcn/ern/default.aspx
This community is administered by the US Africa Command, and anyone worldwide is invited to participate. The community offers maps, file sharing space, translation services, live online chat (including live translation for group chat rooms or private chat), and live news feeds from organizations like the Red Cross, Doctors Without Borders, Relief Web, and more.
To participate, create an APAN account by clicking here and follow the instructions.
Next, visit the Ebola Response Network community on APAN by clicking here, and click “Join Group” as shown below to participate. No approval is necessary.
Humanitarian Assistance and Disaster Response (HA/DR) teams worldwide have benefited from sharing information and collaborating online in APAN communities since January 2010 when earthquakes in Haiti inspired an international and inter-agency response effort that used APAN. To see the many varieties of HA/DR communities built on APAN over the past 5 years, see our Case Studies page here: https://www.apan.org/pages/case-studies.
For assistance with this or any other APAN community, please visit the APAN Support page, submit a Support Ticket via web form, or call the APAN Help Desk directly at (808) 472-7841.
We are going to proceed with our APAN software push tomorrow. We are well aware of the ongoing APAN support operations in Africa, however we will push the software in a method that will keep APAN operational during the entire maintenance period. This software push is important to release prior to support operations in Africa.
Thank you for your patience and understanding.
Jerry G.UISS-APAN Applications Service Provider Technical Dir.
As of 12:00 am Hawaii Standard Time (1000GMT) on Sunday, August 10th, the APAN Help Desk has returned to regular operations following Tropical Storm Iselle. If you need APAN assistance, please contact the APAN Support Hotline at (808) 472-3361 or submit a support ticket.
Starting at 3:30PM Hawaii Standard Time (2330GMT) 07 AUG 2014, the APAN Help Desk will be providing limited support due to Hurricanes Iselle and Julio. If you need assistance, please send an email to email@example.com. We will NOT be able to access our APAN Hotline or APAN Ticket System during the limited support period. If you are experiencing an outage or critical error on APAN, please call the APAN Technician Pager at (808)363-0831 and leave a telephone number that you can be reached at. We anticipate resuming regular APAN Help Desk support on the morning of Sunday, 10 AUG pending the outcome of the hurricanes.
We are pleased to announce to Community Owners the release of APAN Metrics. APAN Metrics provides APAN Community Owners information on Groups and Sites. Community Owners are now able to:
If you are a Community Owner, read the APAN Metrics FAQs and explore various reports specific to your APAN Community with APAN Metrics. If you are not a Community Owner and would like metrics information about a community, contact the owner of that community. For additional assistance, contact the APAN Help Desk at (808) 472-7841 or submit a help ticket at https://community.apan.org/support/p/supportticket.aspx.
APAN is planning routine information assurance patching and software development update changes for the period June through September 2014. When we conduct these routine APAN maintenance and software updates there will be "intermittent" periods of APAN non-availability. We try and plan our routine maintenance and software changes around significant COCOM, agency or organization events, but we don't always get informed of your event dates or times which could lead to us negatively impact your mission or business objectives.
If you want APAN to be aware of your event during June through September 2014, and enable APAN to better plan maintenance and / or software change intermittent outages, please submit an APAN Support Ticket at: https://community.apan.org/support/p/supportticket.aspx with the following information:
Subject: Support RequestAdd Event NameAdd Dates of EventAdd Times of Event (daily, 24X7, etc.)
IF your event is operationally significant enough to your organization to not to have ANY intermittent interruptions due to APAN information assurance patching or software changes, then please identify this in your support request and we will follow-up with your organization to get more information so we can forward your request to higher HQ's.
Thanks for you continued support of APAN.
Jerry GilesUISS-All Partners Access Network (APAN)Applications Service Provider Technical Manager
APAN users will soon have the ability to create, store, and share maps in their communities with features on https://map.apan.org! A preview of APAN’s new mapping service is now available on https://map.apan.org, and additional features will be added in a few weeks.
Map.apan.org provides three main services:
Future expansions of APAN’s mapping services will include:
The homepage for https://map.apan.org will change when these new expansions become available in a few weeks.
After trying APAN’s mapping services preview, send feedback to the APAN team by filling out a Support Ticket here: https://community.apan.org/support/p/supportticket.aspx. User feedback is appreciated!
The APAN homepage has been redesigned to help new APAN visitors and experienced APAN members quickly find information on APAN’s purpose, tools and services, and how APAN can be used to achieve information sharing and collaboration objectives. To get to the new APAN homepage, go to www.apan.org.
Success stories provide examples of how APAN tools have already been applied to real world challenges.
Descriptions of each of the APAN tools are presented for people who are exploring APAN and all it has to offer.
The homepage of APAN Community (https://community.apan.org/) has also been updated to assist users in quickly finding their communities.
For the past 3 months, the APAN Team has been hard at work re-designing the APAN Front Page. We are excited for you to re-discover all the great features APAN has to offer. Until we release the new design at the end of the month, here's a sneak peek at some of the new content:
New mobile friendly design
Case studies highlighting how organizations are using APAN
Details on each of APAN's tools
and so much more!
Translation services on APAN have been expanded! APAN has features to allow users to view the APAN web pages, chat via Instant Message, and translate documents in 17 different languages.
We have simplified the process for selecting your language of choice. Just click the button labeled "English" in the upper left corner of your screen and select your language from the drop down menu. This choice enables the user to view the APAN web pages in their language, chat via Instant Message in their language, and see their language first when they visit Translate.apan.org.
The services offered on https://Translate.apan.org enable users to translate documents and text selections. Users can log into Translate.apan.org with their APAN username and password to get access to a variety of tools that make translating and sharing documents in a multi-lingual team easier.
Click here to learn more about Translate.apan.org in the APAN Knowledge Base.
Hau'oli Makahiki Hou!
The Unclassified Information Sharing Service (UISS) All Partners Access Network (APAN) Application Service Provider team of Software Developers, Knowledge Managers, Project Managers and Computer Systems Engineers would like to wish all our APAN customers a Happy, Healthy and Prosperous New Year. In 2014 APAN will be making some major changes to its front page, navigation and services to include a robust Geo-spatial (GIS) capability, APAN Lite enhancements and forging ahead with APAN mobile app services. We look forward to supporting our customers and consistently improving our unclassified information sharing & collaboration services. Take care and be safe.
APAN Technical Director
A feedback button is now available for users to send feedback to the APAN team! The button can be found on Chat.apan.org, Translate.apan.org, and on the login and account management pages.
Your feedback helps the APAN team improve features and services. User feedback has already helped the APAN team develop the following new features:
The APAN team considers but cannot respond to every feedback submission. For additional assistance, please submit your request in an APAN Support Ticket.