Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets

Jeremy Kepner; Chad Meiners; Chansup Byun; Sarah McGuire; Timothy Davis; William Arcand; Jonathan Bernays; David Bestor; William Bergeron; Vijay Gadepally; Raul Harnasch; Matthew Hubbell; Micheal Houle; Micheal Jones; Andrew Kirby; Anna Klein; Lauren Milechin; Julie Mullen; Andrew Prout; Albert Reuther; Antonio Rosa; Siddharth Samsi; Doug Stetson; Adam Tse; Charles Yee; Peter Michaleas

doi:10.1109/HPEC43674.2020.9286235

Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets

Jeremy Kepner, Chad Meiners, Chansup Byun, Sarah McGuire, Timothy Davis, William Arcand, Jonathan Bernays, David Bestor, William Bergeron, Vijay Gadepally, Raul Harnasch, Matthew Hubbell, Micheal Houle, Micheal Jones, Andrew Kirby, Anna Klein, Lauren Milechin, Julie Mullen, Andrew Prout, Albert ReutherAntonio Rosa, Siddharth Samsi, Doug Stetson, Adam Tse, Charles Yee, Peter Michaleas

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

Our society has never been more dependent on computer networks. Effective utilization of networks requires a detailed understanding of the normal background behaviors of network traffic. Large-scale measurements of networks are computationally challenging. Building on prior work in interactive supercomputing and GraphBLAS hypersparse hierarchical traffic matrices, we have developed an efficient method for computing a wide variety of streaming network quantities on diverse time scales. Applying these methods to 100,000,000,000 anonymized source-destination pairs collected at a network gateway reveals many previously unobserved scaling relationships. These observations provide new insights into normal network background traffic that could be used for anomaly detection, AI feature engineering, and testing theoretical models of streaming networks.

Original language	English
Title of host publication	2020 IEEE High Performance Extreme Computing Conference, HPEC 2020
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781728192192
DOIs	https://doi.org/10.1109/HPEC43674.2020.9286235
State	Published - Sep 22 2020
Externally published	Yes
Event	2020 IEEE High Performance Extreme Computing Conference, HPEC 2020 - Virtual, Waltham, United States Duration: Sep 21 2020 → Sep 25 2020

Publication series

Name	2020 IEEE High Performance Extreme Computing Conference, HPEC 2020

Conference

Conference	2020 IEEE High Performance Extreme Computing Conference, HPEC 2020
Country/Territory	United States
City	Virtual, Waltham
Period	09/21/20 → 09/25/20

Keywords

Internet modeling
hypersparse matrices
packet capture
power-law networks
streaming graphs

Access to Document

10.1109/HPEC43674.2020.9286235

Cite this

Kepner, J., Meiners, C., Byun, C., McGuire, S., Davis, T., Arcand, W., Bernays, J., Bestor, D., Bergeron, W., Gadepally, V., Harnasch, R., Hubbell, M., Houle, M., Jones, M., Kirby, A., Klein, A., Milechin, L., Mullen, J., Prout, A., ... Michaleas, P. (2020). Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets. In 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020 [9286235] (2020 IEEE High Performance Extreme Computing Conference, HPEC 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/HPEC43674.2020.9286235

@inproceedings{89e7cae6bf6942daa1da3bdb2cca910c,

title = "Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets",

abstract = "Our society has never been more dependent on computer networks. Effective utilization of networks requires a detailed understanding of the normal background behaviors of network traffic. Large-scale measurements of networks are computationally challenging. Building on prior work in interactive supercomputing and GraphBLAS hypersparse hierarchical traffic matrices, we have developed an efficient method for computing a wide variety of streaming network quantities on diverse time scales. Applying these methods to 100,000,000,000 anonymized source-destination pairs collected at a network gateway reveals many previously unobserved scaling relationships. These observations provide new insights into normal network background traffic that could be used for anomaly detection, AI feature engineering, and testing theoretical models of streaming networks.",

keywords = "Internet modeling, hypersparse matrices, packet capture, power-law networks, streaming graphs",

author = "Jeremy Kepner and Chad Meiners and Chansup Byun and Sarah McGuire and Timothy Davis and William Arcand and Jonathan Bernays and David Bestor and William Bergeron and Vijay Gadepally and Raul Harnasch and Matthew Hubbell and Micheal Houle and Micheal Jones and Andrew Kirby and Anna Klein and Lauren Milechin and Julie Mullen and Andrew Prout and Albert Reuther and Antonio Rosa and Siddharth Samsi and Doug Stetson and Adam Tse and Charles Yee and Peter Michaleas",

note = "Funding Information: This material is based upon work supported by the Assistant Secretary of Defense for Research and Engineering under Air Force Contract No. FA8702-15-D-0001, National Science Foundation CCF-1533644, and United States Air Force Research Laboratory Cooperative Agreement Number FA8750-19-2-1000. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Assistant Secretary of Defense for Research and Engineering, the National Science Foundation, or the United States Air Force. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. Publisher Copyright: {\textcopyright} 2020 IEEE.; null ; Conference date: 21-09-2020 Through 25-09-2020",

year = "2020",

month = sep,

day = "22",

doi = "10.1109/HPEC43674.2020.9286235",

language = "English",

series = "2020 IEEE High Performance Extreme Computing Conference, HPEC 2020",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2020 IEEE High Performance Extreme Computing Conference, HPEC 2020",

}

Kepner, J, Meiners, C, Byun, C, McGuire, S, Davis, T, Arcand, W, Bernays, J, Bestor, D, Bergeron, W, Gadepally, V, Harnasch, R, Hubbell, M, Houle, M, Jones, M, Kirby, A, Klein, A, Milechin, L, Mullen, J, Prout, A, Reuther, A, Rosa, A, Samsi, S, Stetson, D, Tse, A, Yee, C & Michaleas, P 2020, Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets. in 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020., 9286235, 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020, Institute of Electrical and Electronics Engineers Inc., 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020, Virtual, Waltham, United States, 09/21/20. https://doi.org/10.1109/HPEC43674.2020.9286235

Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets. / Kepner, Jeremy; Meiners, Chad; Byun, Chansup et al.

2020 IEEE High Performance Extreme Computing Conference, HPEC 2020. Institute of Electrical and Electronics Engineers Inc., 2020. 9286235 (2020 IEEE High Performance Extreme Computing Conference, HPEC 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets

AU - Kepner, Jeremy

AU - Meiners, Chad

AU - Byun, Chansup

AU - McGuire, Sarah

AU - Davis, Timothy

AU - Arcand, William

AU - Bernays, Jonathan

AU - Bestor, David

AU - Bergeron, William

AU - Gadepally, Vijay

AU - Harnasch, Raul

AU - Hubbell, Matthew

AU - Houle, Micheal

AU - Jones, Micheal

AU - Kirby, Andrew

AU - Klein, Anna

AU - Milechin, Lauren

AU - Mullen, Julie

AU - Prout, Andrew

AU - Reuther, Albert

AU - Rosa, Antonio

AU - Samsi, Siddharth

AU - Stetson, Doug

AU - Tse, Adam

AU - Yee, Charles

AU - Michaleas, Peter

N1 - Funding Information: This material is based upon work supported by the Assistant Secretary of Defense for Research and Engineering under Air Force Contract No. FA8702-15-D-0001, National Science Foundation CCF-1533644, and United States Air Force Research Laboratory Cooperative Agreement Number FA8750-19-2-1000. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Assistant Secretary of Defense for Research and Engineering, the National Science Foundation, or the United States Air Force. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. Publisher Copyright: © 2020 IEEE.

PY - 2020/9/22

Y1 - 2020/9/22

N2 - Our society has never been more dependent on computer networks. Effective utilization of networks requires a detailed understanding of the normal background behaviors of network traffic. Large-scale measurements of networks are computationally challenging. Building on prior work in interactive supercomputing and GraphBLAS hypersparse hierarchical traffic matrices, we have developed an efficient method for computing a wide variety of streaming network quantities on diverse time scales. Applying these methods to 100,000,000,000 anonymized source-destination pairs collected at a network gateway reveals many previously unobserved scaling relationships. These observations provide new insights into normal network background traffic that could be used for anomaly detection, AI feature engineering, and testing theoretical models of streaming networks.

AB - Our society has never been more dependent on computer networks. Effective utilization of networks requires a detailed understanding of the normal background behaviors of network traffic. Large-scale measurements of networks are computationally challenging. Building on prior work in interactive supercomputing and GraphBLAS hypersparse hierarchical traffic matrices, we have developed an efficient method for computing a wide variety of streaming network quantities on diverse time scales. Applying these methods to 100,000,000,000 anonymized source-destination pairs collected at a network gateway reveals many previously unobserved scaling relationships. These observations provide new insights into normal network background traffic that could be used for anomaly detection, AI feature engineering, and testing theoretical models of streaming networks.

KW - Internet modeling

KW - hypersparse matrices

KW - packet capture

KW - power-law networks

KW - streaming graphs

UR - http://www.scopus.com/inward/record.url?scp=85099367366&partnerID=8YFLogxK

U2 - 10.1109/HPEC43674.2020.9286235

DO - 10.1109/HPEC43674.2020.9286235

M3 - Conference contribution

AN - SCOPUS:85099367366

T3 - 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020

BT - 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 21 September 2020 through 25 September 2020

ER -

Kepner J, Meiners C, Byun C, McGuire S, Davis T, Arcand W et al. Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets. In 2020 IEEE High Performance Extreme Computing Conference, HPEC 2020. Institute of Electrical and Electronics Engineers Inc. 2020. 9286235. (2020 IEEE High Performance Extreme Computing Conference, HPEC 2020). doi: 10.1109/HPEC43674.2020.9286235

Multi-Temporal Analysis and Scaling Relations of 100,000,000,000 Network Packets

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this