|The only thing one knows about human nature is that it changes. - Oscar Wilde|
C.3 Engineering Plan
The yet early stage of development of Russian high performance networking limits the detail which can be provided regarding timeline, service metrics and exact network topology. In addition to meeting its goals of fostering US-Russian collaborative effort over the high performance network, the NaukaNet consortium will be able to cooperatively develop a leading edge, high performance Russian academic network. The following describes the initial engineering plan for fostering its development.
C.3.1 Description of the NaukaNet Network
The following figure illustrates the proposed NaukaNet infrastructure connecting the research network in Moscow to STAR TAP in Chicago to vBNS and other HPIIS organizations.
Figure 2: NaukaNet Topology
A 6Mbps terrestrial based connection will be provided by Ameritech (Note 1) from the Chicago NAP to Moscow. A PVP will be established between the STAR TAP ATM switch and the ATM switch in the M9 station in Moscow. At this time, STAR TAP only offers layer 2 ATM service, therefore a PVC with UBR service type will be defined inside the tunnel to connect the NaukaNet router to the peering vBNS router to provide layer 3 IP services. Routing information will be exchanged between peering routers via BGPv4. Future peering connections will be established in the same fashion over the STAR TAP ATM switch.
In order to provide native ATM-based service, a set of UBR PVCs will be defined between the STAR TAP and the M9 station with the intention of allocating and scheduling the use of these connections by NaukaNet. The choice of using UBR service at this time is to allow for maximum flexibility and efficiency (to minimize responsibility from such entities as STAR TAP/Ameritech (Note 1) in setting up connections) until ABR is ready. Real-time as well as historical data of usage statistics on the NaukaNet link (cell rate per VP or VC, network traffic flow, protocol type distribution) will be used to evaluate link efficiency. Regularly scheduled link throughput testing will be used for quality assurance purposes.
C.3.2 Interoperability of NaukaNet and the US VBNS
NaukaNet's connection to STAR TAP, which will be from RBnet facilities in Russia, will have its own autonomous network (AS) number. It is the IPv4 addressing authority for the RBnet network. All HPIIS approved institutions will be connected layer 3 IP-wise to RBnet which in turn will provide the transport to STAR TAP based on the policies defined in the NaukaNet policy router.
Router peering between vBNS and NaukaNet will be established via BGPv4, over a PVC defined on the ATM switches end-to-end (STAR TAP and M9). The vBNS router will advertise routes for all vBNS nodes to the HPIIS policy router in Moscow. HPIIS router in Moscow will advertise only those routes from HPIIS authorized institutions to the vBNS router by performing route filtering. Additional router peering connections to other HPIIS authorized institutions at STAR TAP will be accomplished via similar fashion.
ATM layer-2 services from the emerging research network to vBNS or HPIIS authorized nodes will be accomplished via predefined PVCs with assistance from vBNS and STAR TAP (if necessary). Scheduling of such connection will be coordinated by the NaukaNet NOCs at RBnet and UTK.
C.3.3 Other Issues
NaukaNet recognizes that network security is a important aspect of any robust network. Therefore, NaukaNet will deploy the necessary monitoring and administration equipment to prevent potential security problems.
On the policy router at RBnet, a set of filters will be defined to prevent traffic on following TCP/UDP ports (incoming connection from STAR TAP) that are known to have been exploited in security breaches/attacks on other networks:
Exceptions to these filters will be dealt with by the NaukaNet NOCs and traffic on these ports can be permitted on an individual basis. The policy router that will be deployed at RBnet will be capable of permitting traffic from certain hosts using the ports listed to pass through, while denying others.
In addition, filters will be defined on the policy router at RBnet to prevent IP spoofing attacks.
Attempts to generate traffic as listed above will be logged by the policy router via SYSLOG facility (outgoing) to a secure station in the NaukaNet NOC in Russia as well as in the US for investigation.
To prevent potential route leaks from other institutions, the policy router at RBnet will be configured to only accept updates for networks that are vBNS authorized and HPIIS authorized (in the future).
NaukaNet will also cooperate with the vBNS NOC as well as other partner HPIIS institutions to define necessary filters to prevent routing leaks, network attacks, IP spoofing from NaukaNet.
All information gathered in monitoring for network security violations will remain confidential unless requested by the appropriate authority. No target monitoring of a specific user will take place without the appropriate authorities.
NaukaNet is designed to avoid single point of failure. The domestic portion of the link is backed up by the Ameritech (Note 1) SONET ring between New York and Chicago. Backup on the transatlantic link is provided by the restoration agreements between CANTAT-3 and TAT12. Service level agreements offered by Ameritech (Note 1) for ATM transport level service is +99.9% of uptime. Ameritech (Note 1) has traditionally provided a very robust service to its customers. The uptime for the Chicago NAP was 99.998% for the last four years with the longest outage over this four year period being only 2 hours.
All ATM switches and policy routers are built with redundant and hot-swappable power supply, cooling-fans, processor modules and backup ports. Uninterruptable Power Supplies (UPS) will be installed for each component with at least an hour of stand-by time to achieve maximum reliability. All facilities will be located in the provider (of the proposed link) switch room, or installed in an environmentally sound location.
C.3.3.3 Advanced Services
NaukaNet is committed to implementing advanced network services whenever possible. Specifically, NaukaNet will research and implement the following when feasible:
UT already has extensive experience in the use of hierarchical web-caching. It currently participates in the caching scheme developed by NLANR using the Squid cache. NaukaNet will implement a similar caching scheme at the NaukaNet network in Moscow for WWW sites maintained on vBNS and other HPIIS institutions. This should significantly reduce WWW related traffic over the transatlantic link as vBNS contains a relatively small set of WWW sites. The effectiveness of the caching server will only be increased.
UT also has experience with IPv6 and is connected to the 6-bone via ORNL. UT will leverage this experience and knowledge to promote the adoption of IPv6 throughout NaukaNet.
C.4 Operations, Monitoring, and Quality Assurance Plan
The operation, monitoring and quality assurance will be a shared responsibility between the NaukaNet NOCs at UT and RBnet. The NaukaNet NOC will work with Ameritech (Note 1), as well as STAR TAP in providing a robust network connection.
Ameritech (Note 1) will be held responsible for the integrity as well as the quality of the proposed link. Ameritech (Note 1) will be the point of contact for any problems attributable to any portion of the proposed link at the transport or physical level.
The NOCs at UT and RBnet will be primarily responsible for the following tasks:
The NaukaNet NOCs will interface with the Network Information Centers in providing reports and feedback to the user community. For additional information on the reporting aspect of NaukaNet, please see section C.6 Reporting and Publications.
Coordination between the NOCs in Russia and the US will be accomplished by several methods. Web based chat tools will be used as the primary means of conversation/coordination between NaukaNet NOCs. The benefit of this method of communication is that conversations that take place during the chat sessions can be archived for review at another time. Each NOC will maintain the common chat screen on a 24x7 basis. Additional lines of communication between the NOCs will include desktop video conferencing as well as conversations over the traditional telephone when the situation requires. The use of video conferencing will be rare in the interest of preserving network bandwidth for application traffic.
In order to coordinate trouble shooting between the NOCs at RBnet and UTK, language and cultural differences must be addressed. UTK currently has on staff one Russian native and one Russian speaking American that will assist in the setup and future trouble shooting of the HPIIS link. Additional Russian nationals will be placed as needed.
The NaukaNet NOC in Russia will be responsible for gathering interface statistics from the policy router in Moscow. Specifically, the following interface counters will be polled via the Simple Network Management Protocol (SNMP) on a five-minute interval:
In addition, router CPU load will be polled on a five-minute interval as well. Statistics gathered will be published via the WWW at F&P Russia as well as the NIC in the US (UT) in real time to provide user feedback. UT has already developed such tools and is currently using these tools to manage its campus network.
The NaukaNet NOC in Russia will also gather network flow statistics that the policy router is capable of providing. The NaukaNet NOC will be able to identify traffic flow patterns, packets and bytes per flow, incoming and outgoing TCP/UDP ports to obtain the traffic characteristics on NaukaNet. This information will be archived for historical analysis. NaukaNet will institute procedures to maintain the confidentiality of the information gathered by using such program that will substitute the source and destination IP of the packet header in the log file. The TNS department at UT has already developed tools based on this type of data to identify potential security problems as well as traffic flow patterns.
The NaukaNet NOC in the US will be responsible for monitoring switch statistics at the STAR TAP. At this time, STAR TAP provides per port statistics (Average/Max Input/Output Mbps) on a 5 minutes interval. NaukaNet plans to deploy an OC3MON (developed by MOAT at NLANR) monitoring station at STAR TAP in order to obtain additional information regarding to traffic characteristics. As part of this deployment, a LS1010 provided by NaukaNet will be co-located at the STAR TAP for media conversion of DS3/OC3 conversation until the installation of the Stratacom switch at the STAR TAP.
The NaukaNet NOC in the US and Russia will be jointly responsible for the monitoring of routing stability on NaukaNet. Upon peering with vBNS, historical routing tables will be kept daily/weekly in locating new routes in the routing table as well as identifying potential routing abnormalities.
The NaukaNet NOC in the US and Russia will be jointly responsible for the monitoring of potential security problems on NaukaNet. The NaukaNet router will be configured with two secure hosts (one each in the NOCs in Russia and US) for the logging of all informational (including critical) events and messages. This log will be monitored real time for immediate response to potential problems.
NaukaNet will form a partnership with existing organizations such as the Cooperative Association for Internet Data Analysis (CAIDA), funded by NSF, to promote greater cooperation in the engineering and maintenance of a robust, scaleable global Internet infrastructure.
IPv4 based network service on NaukaNet is provided by the peering of the policy router at RBnet with vBNS or other HPIIS approved institutions. Traffic of this nature will take place on the defined PVC(s) between peering routers with UBR service level. By the nature of the UBR service type, the amount of total bandwidth is the upper limit of the PVC(s). Typically there is no scheduling necessary for this type of traffic.
Native ATM layer 2 service will be provided by request only. Upon receiving of the request, the NaukaNet NOC will either allocate a predefined PVC (UBR service) to the researcher at the time requested, or coordinate with STAR TAP and vBNS or other members involved in establishing the connection with the required service. The NaukaNet Consortium will establish a shared database and a procedure to accommodate scheduling and to resolve scheduling conflicts.
NaukaNet will, with cooperation from STAR TAP and other institutions, aggressively experiment/incorporate other type of services such as IPv6 and available bit rate (ABR) as these technologies mature.
Scheduling of network events/outage will be posted at least one week (7 days) in advance unless a situation requires emergency repairs. The NaukaNet consortium will maintain a LISTSERV discussion group for all interested members of the user community for this purpose. Such events and notices will be posted via WWW at the NaukaNet web site as well.
C.4.3 Network Troubleshooting
The NaukaNet NOCs will be responsible for the initiation of all trouble tickets for the proposed connection, whether the problem is reported directly by the user, or identified by the monitoring personnel at the NOC. The reporting of the problem can take place in the form of an email message, telephone call, or submission of web-based form at the NaukaNet site.
Researchers and users in Russia will directly report problems regarding the proposed connection to the NaukaNet NOC at RBnet. The NaukaNet NOC in the US will handle all reports of problems by users on vBNS or other HPIIS authorized institutions. Although the two NOCs will each maintain a separate trouble ticketing system, these ticketing systems should be of the same configuration for sharing of the trouble ticket database.
Upon the creation of a ticket under each NOC, the other NOC should be immediately notified in the form of the email, or in the case of a link down, be communicated over the telephone. Each NOC will be responsible for keeping track of all trouble tickets created in its system. All problems will be analyzed and results reported and shared between the two NOCs. Problems related to the proposed link will be posted to the WWW site maintained by NaukaNet.
Reporting of network events and statistics will take place in two forms. Notification of network events/outage will be posted via email to the LISTSERV mailing list managed by NaukaNet as well as published on the web at NaukaNet.
Network statistics will be gathered by the NaukaNet NOC at RBnet and made available on WWW except for statistics related to a particular application / experimentation or deemed non-public. These statistics will only be furnished to the researcher/user that owns the application. All statistics will be mirrored at the NaukaNet NIC in US for perusal by users in the US in order to save the trans-continental bandwidth.
C.4.5 Quality Assurance
The measure of quality of the proposed NaukaNet link will be based on several criteria. The number one measurement will be the uptime of the link (including the transport layer). At least 99% uptime is desired by the NaukaNet consortium over a one year period for this link to be successful (this does not include scheduled outages for upgrades and new service implementations). The next biggest measurement unit for the success of NaukaNet will be based on the number of application it supports the quality of the applications, and the tangible results obtained through the implementation. Additional measurement units will be based on user feedback from researchers of HPIIS institutions.
C.4.6 Policy Enforcement
The NaukaNet implementation will follow the guidelines set forth by NSF in allowing only HPIIS authorized traffic to utilize the proposed link. This will be largely accomplished by implementing route-map and route filtering (policy based routing) on the policy router in Moscow. The NaukaNet NOCs in both the US and Moscow will each maintain copies of router configuration and full routing table on the policy router. Daily comparisons will be made between routing table logged in successive days in detecting routing abnormalities. The policy router on NaukaNet will only connect to HPIIS authorized institutions in Russia to further reduce the potential of the violation of NSF policies regarding HPIIS connections. Policy enforcement over layer 2 ATM service will be less of an issue because such connections are only provided by advance scheduling. When SVC services are offered at STAR TAP, additional procedures will be developed in the acceptable usage policy enforcement.
C.4.7 NaukaNet Network Information Center
Described under C.6 Reporting below.
C.4.8 NaukaNet Management Structure
Described under C.5.7 Management Structure.
Funding for NaukaNet provided by the US National Science Foundation and the
Ministry for Industry, Science and Technology of the Russian Federation.
Telecommunications services are provided by Telia, Inc.
This NaukaNet web site is available at two locations, in US and Russia:
This NaukaNet web site is available at two locations, in US and Russia:
Please contact the
with your comments and suggestions.