From Harald.Alvestrand@maxware.no Sat Jun 20 15:56:48 1998 Received: from alden ([10.128.1.78]) by dokka.maxware.no (8.8.5/8.8.5) with SMTP id PAA10103 for ; Sat, 20 Jun 1998 15:56:48 +0200 Message-Id: <3.0.2.32.19980620155921.00a659b0@dokka.maxware.no> X-Sender: hta@dokka.maxware.no X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.2 (32) Date: Sat, 20 Jun 1998 15:59:21 +0200 To: bmwg-archive@alvestrand.no From: Harald Tveit Alvestrand Subject: Test message Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Just a test of the BMWG archiving facility -- Harald Tveit Alvestrand, Maxware, Norway Harald.Alvestrand@maxware.no From owner-bmwg@BayNetworks.COM Wed Jun 24 23:06:11 1998 Received: from smtp-gw.BayNetworks.COM (ns1.BayNetworks.COM [134.177.3.20]) by dokka.maxware.no (8.8.5/8.8.5) with ESMTP id XAA07059 for ; Wed, 24 Jun 1998 23:06:09 +0200 Received: from mailhost.BayNetworks.COM (screen2r.BayNetworks.COM [134.177.3.1]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id OAA24243 for ; Wed, 24 Jun 1998 14:07:14 -0700 (PDT) Received: from pobox.engeast.BayNetworks.COM (pobox.engeast.baynetworks.com [192.32.61.6]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id OAA18835 for ; Wed, 24 Jun 1998 14:07:21 -0700 (PDT) Received: (majordom@localhost) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) id RAA29161; Wed, 24 Jun 1998 17:07:17 -0400 for Date: Wed, 24 Jun 1998 17:07:17 -0400 Message-Id: <199806242107.RAA29161@pobox.engeast.BayNetworks.COM> To: bmwg-archive@alvestrand.no From: majordom@BayNetworks.COM Subject: Welcome to bmwg Reply-To: majordom@BayNetworks.COM -- Welcome to the bmwg mailing list! If you ever want to remove yourself from this mailing list, you can send mail to "majordom" with the following command in the body of your email message: unsubscribe bmwg bmwg-archive@alvestrand.no Here's the general information for the list you've subscribed to, in case you don't already have it: ********************************************************************* IETF Benchmarking Methodology Working Group Mailing List. For more info, see http://www.ietf.org/html.charters/bmwg-charter.html ********************************************************************* From kdubray@BayNetworks.COM Wed Jun 24 23:52:39 1998 Received: from smtp-gw.BayNetworks.COM (ns1.BayNetworks.COM [134.177.3.20]) by dokka.maxware.no (8.8.5/8.8.5) with ESMTP id XAA07479 for ; Wed, 24 Jun 1998 23:52:37 +0200 Received: from mailhost.BayNetworks.COM (screen2r.BayNetworks.COM [134.177.3.1]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id OAA29626 for ; Wed, 24 Jun 1998 14:53:44 -0700 (PDT) Received: from pobox.engeast.BayNetworks.COM (pobox.engeast.baynetworks.com [192.32.61.6]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id OAA21759 for ; Wed, 24 Jun 1998 14:53:53 -0700 (PDT) Received: from pestilence.engeast (pestilence [192.32.134.71]) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) with SMTP id RAA02942; Wed, 24 Jun 1998 17:53:49 -0400 for Date: Wed, 24 Jun 1998 17:53:49 -0400 From: kdubray@BayNetworks.COM (Kevin Dubray) Message-Id: <199806242153.RAA02942@pobox.engeast.BayNetworks.COM> To: bmwg-archive@alvestrand.no Subject: Revised BMWG Minutes Minutes of the Benchmarking Methodology Working Group (BMWG) [These minutes have been edited.] Reported by Kevin Dubray Kevin Dubray opened the Benchmarking Methodology working group session. Upon regarding the light turnout (about 30) he commented that holding a session in the last meeting slot on the last day before the Plenary might not be such a good thing... Light turnout notwithstanding, Dubray presented the agenda for the 41st IETF BMWG meeting. Agenda: 1. Administration 1.1 Agenda Bashing 1.2 RFC 2285 2. Benchmarking Terminology of Firewall Performance 3. Terminology for IP Multicast Benchmarking 4. Goals for Next Period. 1. Administration. Dubray announced that this would be an abbreviated session since unexpected scheduling changes forced some speakers to amend their plans. David Newman asked for an item on the agenda to be added to ascertain interest in a new work item. The chair amended the agenda to reflect this change. The chair announced that the work formerly known as the LAN Switching Benchmarking Terminology draft was approved by the IESG as an Informational Request for Comments. The RFC editor designated this work as RFC 2285. Dubray thanked the editor, Bob Mandeville, and the other people who had contributed to the draft. 2. Benchmarking Terminology of Firewall Performance David Newman was introduced to lead a discussion on the Firewall Performance Benchmarking draft. David began with a quick overview of the changes to the draft. He describe the two major changes revolving around the notion of: A. Connection B. Forwarding Rate Newman then engaged in a detailed discussion of the changes to this latest version of the draft. He noted that the term "connection" was changed pursuant to the comments at the Washington DC IETF. He noted that the term "session" was modified as well. David noted that given the relationship between the two terms, it might be a good idea to combine the two at some point. He went on to say that the concept of "external" was modified to reflect the idea of an "unprotected" network segment. David said the draft also offered a definition for the term, "Firewall." David conveyed that because of value-added features sometimes supplemented to firewalls, the definition might not seem encompassing. These supplemental features nothwithstanding, David feels the firewall definition is complete and historically accurate. In presenting the term "forwarding rate," Newman articulated that he believed it was significantly different than the term in RFC 2285. A comment was offered that, at first glance, the only seemingly different notions were that of the measurement units "bits" versus "frames" per second and that of the more specific term "firewall" in lieu of "DUT/SUT". The follow-up question then asked how different were the notions of "Forwarding Rate" in this draft over RFC 2285? Newman thought that they were substantially different - a frame-based metric is not meaningful in the context of an application layer device like a firewall. A trailing statement remarked that the two terms were ambiguous outside the context of the separate discussion. Newman acknowledged that the overlap was troubling. He asked for suggestions. One suggestion offered changing the name to something more firewall specific - perhaps a "Firewall Forwarding Rate" metric. Another suggestion was made to align the metric's title closer to the primary difference, "bit forwarding." This was met with general approval excepting a caveat that the term not necessarily be restricted to just firewalls. David asked for clarification of what was the consensus, firewall forwarding rate or bit forwarding rate. The generic bit-forwarding term was reaffirmed. Upon visiting the term "Session," a discussion ensued regarding the differentiation between the terms session and connection. Dubray commented that earlier in the session, Newman referred to Session as a State - "data flowing" - though the term "state" was not expressly used in the draft. Dubray thought it might be useful to explicitly define session as a state. For example, "A state in which higher level transactions can occur or is occurring between associated peers." It was further noted that Newman verbally referred to a connection as a state, too. The draft, however, refers to connection not as a state, but as a "path." A question was posed as to whether the draft, in defining connection and session, was attempting to define a hierarchy between a data path and data transfer state. The example was offered that many sessions could be offered over a single connection. David reiterated his view that a "connection," as defined, was as state - communicating a hierarchy was not a goal. David expressed that he viewed connection as a table entry in a device. [At the meeting's conclusion, Kim Martin offered that a single, unifying concept - circuit, could be perhaps used. It's historical use is consistent to the tone of the discussion. This was noted by Mr. Newman as a good idea.] David then moved the discussion to the modifications surrounding stateful inspection versus stateful packet filtering. He noted the wording was changed, per the DC meeting, to be vendor-neutral. As the discussion moved on to security considerations, a question was offered regarding the merits of assessing firewall functionality. David strongly stated that is was not the draft's intention to provide device validation or other verification-style tests. A voice was heard to say that an understanding could not be gathered on whether performance may compromise functionality. A general question was poised to Mr. Newman. The query went along the lines of what level did Mr. Newman perceived the end to end testing? Layer 3? Layer 4? Firewall functionality vs. Forwarding path? David communicated, "all the above." In closing the discussion, final questions were offered Mr. Newman. In his opening remarks, Mr. Newman referred to two major metrics of the draft: a) forwarding rate, and b) connection. Forwarding rate was straightforward, but how does one make a metric of a data path or state? David said that you could measure the number of concurrent connections. The person posing the question articulated that number of connections seems more like a measurement unit than a metric. Using connection as a metric may be ambiguous. David acknowledged the comment. David thanked the group for its input. 3. Terminology for IP Multicast Benchmarking Kevin Dubray began the discussion of with a quick overview of the draft's initiatives and areas of concentration. He noted that the review at this meeting marked the coverage of all the metrics presented by the Multicast Terminology draft. He then gave a high level summary of changes to the current draft. Two areas of change Dubray focused on was a) the notion of "interaction," and b) the fact that "Fairness" was removed from the draft. In regards to the latter, Dubray articulated that Fairness was removed from the draft in a direct response to queries regarding QoS as being a possible area of future BMWG work. The section on Interaction was added as a direct response to input from the Washington, D.C. BMWG meeting and input from the BMWG mailing list. Dubray presented the metric Encapsulation Throughput. Dubray indicated that Decapsulation Throughput and Re-encapsulation throughput were defined similarly. When a question was posed as to why three separate metrics were needed, Kevin pointed out that these individual metrics were presented as one metric, Translational Throughput in the last draft. Dubray said it was intended that the methodology be the defining factor. Dubray indicated that the working group thought it would be better to define three discrete metrics. The members of this session reaffirmed the previous wisdom. With old business behind, Dubray ensued in a discussion of the last sections of the draft: Overhead, Capacity, and Interaction. Dubray indicated that feedback from the mailing list indicated three thoughts, not necessarily all predominant: A. Additional factors contributing to this metric should be identified. B. Crisper definition as to start/stop clocking-type events. C. Join/Leave Delay metrics are more "interesting" in the presence of other factors, such as Offered Load. D. The notion of "Time duration" is generically vague. Dubray presented the wording to relieve the condition cited in A. The group offered no dissenting views. Dubray communicated his thoughts that items B and D were better suited for the corresponding methodology document. The group agreed. He thought that the associated "burdened response" metric to be presented later would raise the interest index cited in condition C. On Group Join Delay (3.4.1), there was a question as to what constitutes the successful transaction. Kevin thought it was a good question - but, again, thought it was a question better answered in the methodology document. The group agreed. The group further concurred that the additions of Interaction (3.6) and Burdened Response (3.6.1) were welcome to the draft. On the topic of Forwarding-Burdened Multicast Latency, there was a question regarding the nature of the metric. Was the goal to parameterized the latency metric? Dubray said yes, especially with respect to load and associated forwarding rate. An additional question queried as to what should be gleaned from the metric. Dubray indicated the intent was not to limit the statistical information obtained from the measurement. So, from the set of n latencies, the appropriate statistics could be collected. There was little discussion on the remaining part of the presentation. With that, Dubray announced that the major parts of the draft had been presented and discussed. He asked if there were any issues. A question was raised on the Multicast Group Capacity metric. While the metric tried to identify the maximum number of multicast groups a device can support, it did so with the qualification that traffic be correctly forwarded. Specifically, what level/rate of traffic needed to be processed to indicate goodness. Dubray indicated that he didn't know. Moreover, he didn't think the actual rate was important. He thought it was more important to ascertain the correct delivery. Regardless, he thought it was an issue best addressed by the corresponding methodology document. Dubray asked that if the changes suggested were made, would the document be ready for final review. There were no dissenting reviews. Dubray then invited David Newman to present the new work item. David Newman said that he wished to make a proposal for Bob Mandeville, who was unable to make the meeting. David informed the group that Bob was hoping to do some work in the QoS area using latency as vehicle. David asked if there was interest. The group indicated that there was. Someone suggested that the work would be more useful if it were expanded beyond latency. Others agreed. The chair asked that Mandeville submit a workplan for WG consideration. Interest was again indicated in the area of the Cell/Call benchmarking draft. Dubray indicated that Raj Jain had hoped to attend, but he was called away at last minute. Dubray indicated that he anticipated a rise in activity in this direction. In the interim, he asked that folks start to open the discussion on the BMWG mailing list. With no new business, Dubray reviewed the next period's goals: 1. Advance the Firewall Draft; get it in the shape for final review. 2. Prepare the Multicast draft for final review. 3. Deliver a LAN Switch Methodology draft to be a companion document to RFC 2285. 4. Revise and resubmit a Call/Cell terminology draft. 5. Review a QoS benchmarking workplan, if submitted. From owner-bmwg@BayNetworks.COM Thu Jun 25 00:10:24 1998 Received: from smtp-gw.BayNetworks.COM (ns1.BayNetworks.COM [134.177.3.20]) by dokka.maxware.no (8.8.5/8.8.5) with ESMTP id AAA07692 for ; Thu, 25 Jun 1998 00:10:23 +0200 Received: from mailhost.BayNetworks.COM (screen2r.BayNetworks.COM [134.177.3.1]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id PAA00953; Wed, 24 Jun 1998 15:04:19 -0700 (PDT) Received: from pobox.engeast.BayNetworks.COM (pobox.engeast.baynetworks.com [192.32.61.6]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id PAA22478; Wed, 24 Jun 1998 15:04:23 -0700 (PDT) Received: (majordom@localhost) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) id SAA03510; Wed, 24 Jun 1998 18:00:17 -0400 for bmwg-outgoing Received: from pestilence.engeast (pestilence [192.32.134.71]) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) with SMTP id SAA03504; Wed, 24 Jun 1998 18:00:16 -0400 for Date: Wed, 24 Jun 1998 18:00:16 -0400 From: kdubray@BayNetworks.COM (Kevin Dubray) Message-Id: <199806242200.SAA03504@pobox.engeast.BayNetworks.COM> To: bmwg@BayNetworks.COM Subject: Administrivia Sender: owner-bmwg@BayNetworks.COM Precedence: bulk Two things: 1. The BMWG archive has moved: http://www.alvestrand.no/archives/bmwg Thanks to Harald Alvestrand for fixing this dilemma. 2. The revised BMWG minutes from the 41st BMWG are not accurately reflected in the online IETF proceedings. The corrected minutes can be accessed via the archive mentioned above. From owner-bmwg@BayNetworks.COM Thu Jun 25 16:07:08 1998 Received: from smtp-gw.BayNetworks.COM (ns1.BayNetworks.COM [134.177.3.20]) by dokka.maxware.no (8.8.5/8.8.5) with ESMTP id QAA16332 for ; Thu, 25 Jun 1998 16:07:03 +0200 Received: from mailhost.BayNetworks.COM (screen2r.BayNetworks.COM [134.177.3.1]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id HAA20092; Thu, 25 Jun 1998 07:01:01 -0700 (PDT) Received: from pobox.engeast.BayNetworks.COM (pobox.engeast.baynetworks.com [192.32.61.6]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id HAA24765; Thu, 25 Jun 1998 07:00:55 -0700 (PDT) Received: (majordom@localhost) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) id JAA15580; Thu, 25 Jun 1998 09:55:05 -0400 for bmwg-outgoing Received: from pestilence.engeast (pestilence [192.32.134.71]) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) with SMTP id JAA15575; Thu, 25 Jun 1998 09:55:04 -0400 for Date: Thu, 25 Jun 1998 09:55:04 -0400 From: kdubray@BayNetworks.COM (Kevin Dubray) Message-Id: <199806251355.JAA15575@pobox.engeast.BayNetworks.COM> To: bmwg@BayNetworks.COM Subject: RE:Thoughts on firewall performance draft Sender: owner-bmwg@BayNetworks.COM Precedence: bulk From: Edward Gonen To: "'Firewall Performance Group'" Subject: RE:Thoughts on firewall performance draft Date: Thu, 25 Jun 1998 10:45:47 +0200 Return-Receipt-To: Edward Gonen Dear > BMWGers, Could you please add me to this mailing list? Below I repost my comments and David's answers regarding performance draft. There was a kind of misunderstanding - I thought we are talking about methodology but as David Newman explained it is rather terminology definition stage so I'll keep my comments until we get to the methodology. Edward Gonen NetGuard. Ltd. Edward@guardian.co.il Thanks very much for your thoughts, Edward. By the way I would strongly encourage you to repost this to the bmwg mailing list, if you're not already on the list. I've commented in ((double parentheses)) on a few points below. dn Edward Gonen > on 06/24/98 05:28:31 AM To: David Newman/NYC/CMPNotes, "'kdubray@BayNetworks.COM'" > Subject: RE:Thoughts on firewall performance draft David, I've got your mail "Thoughts on firewall performance draft" somehow forwarded to me and here are some points I'd like to mention: > > Bit forwarding rate. > The number of bits per second of allowed traffic a firewall can be > observed > to >transmit >successfully to the correct destination interface in > response > to a specified offered load. Yes this definition is better than "frames" or "packets". But how do you calculate this number? If we are talking about TCP/IP then there are MAC header, IP header, TCP, UDP ... and eventually - the data. Some of the performance benchmarks measure only the application data size (HTML page size or FTP files), while others calculate total traffic including MAC header. In my opinion only the data part (what comes after TCP / UDP headers) of the TCP or UDP should be included. Of course with packets ~1Kbyte you may not count the header but when the number of packets is huge and they are smaller the addition of headers is quite important. ((I don't agree that only the payload should be counted. This may reflect my bias toward router and switch-type benchmarks, but remember that a firewall has to process the entire datagram, including header. As you note this may be a statistically significant amount of work, especially with smaller packets. Since the header comprises part of the workload handled by the firewall, I feel it should be counted in measuring forwarding rate.)) Another thing - traffic direction. In most of the benchmark tests there is device under test (firewall) installed between a large number of clients and a smaller number of servers thus the traffic is mostly one-way road - most of the data is coming from the servers toward the clients. In my opinion in the test environment the clients and servers (receivers and senders) should be also swept places. The result may be calculated as an average value of both tests. There is also question regarding which traffic is measured - UDP or TCP. In case of TCP there is an issue of acknowledges sent in the opposite direction. But we can ignore that if the clients and servers are swept. ((Can you please define the term "swept places"? I'm not familiar with it. Past versions of the draft had included definitions of unidirectional and bidirectional traffic, but the consensus within the bmwg was that the definitions were not sufficiently different from existing definitions to merit inclusion here. This purpose of this memo is to describe *terminology* used in benchmarking firewall performance. The bmwg generally follows a two-step process, with a terminology document first and a methodology second. Traffic direction is very germane to the methodology document.)) Another issue is which services are used in the test. I think that performance of the application-proxy firewall depends on which service is used - HTTP, FTP, SMTP etc. The differences may be very noticeable. ((Indeed. I've included definitions of proxies in this terminology document. Describing which proxies are invoked is a methodology issue, however.)) Now is another point - how the firewall should be prepared to the tests. I think it is a good idea that (like in the past tests) vendor's representatives should prepare the firewall for performance tests, but it is crucial that after that preparation and before the performance tests, the firewall is tested for security. It must be proven that there is no security degradation when the firewall is configured for the performance tests. It may be also a good idea to perform the security tests even after the performance tests. ((Again, this is largely a methodology issue. This is going to be a very difficult issue to sort out. The task of "proving there's no security degradation" is very open-ended, to say the least. The terminology document only contemplates that a given rule set is enforced. What about security issues outside the rule set, like the use or nonuse of NAT and hardening of the device against various denial-of-service attacks? I don't have an answer to this as yet and would welcome your thoughts.)) Next time I'll write about connections and sessions.. ((I'd very much appreciate your input on this as well.)) I'll be glad to hear your opinions. Edward Gonen Vice President of R&D NetGuard Ltd. ================================================================ eMail: edward@guardian.co.il > Phone:+972-6-6449944 Mobile:+972-52661113 Fax:+972-6-6540124 URL:http://www.ntguard.com > Address: LanOptics Building, P.O.Box 184, Ramat Gabriel Industrial Park, Migdal Ha-Emek 10551, Israel From owner-bmwg@BayNetworks.COM Fri Jun 26 21:46:26 1998 Received: from smtp-gw.BayNetworks.COM (ns5.baynetworks.com [194.133.90.101]) by dokka.maxware.no (8.8.5/8.8.5) with ESMTP id VAA03005 for ; Fri, 26 Jun 1998 21:46:09 +0200 Received: from mailhost.BayNetworks.COM ([132.245.135.115]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id VAA17503; Fri, 26 Jun 1998 21:33:14 +0200 (MET DST) Received: from pobox.engeast.BayNetworks.COM (pobox.engeast.baynetworks.com [192.32.61.6]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id PAA16211; Fri, 26 Jun 1998 15:31:48 -0400 (EDT) Received: (majordom@localhost) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) id PAA26662; Fri, 26 Jun 1998 15:25:01 -0400 for bmwg-outgoing Received: from pestilence.engeast (pestilence [192.32.134.71]) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) with SMTP id PAA26657; Fri, 26 Jun 1998 15:25:00 -0400 for Date: Fri, 26 Jun 1998 15:25:00 -0400 From: kdubray@BayNetworks.COM (Kevin Dubray) Message-Id: <199806261925.PAA26657@pobox.engeast.BayNetworks.COM> To: bmwg@BayNetworks.COM Subject: Re: Thoughts on firewall performance draft Sender: owner-bmwg@BayNetworks.COM Precedence: bulk From dnewman@data.com Fri Jun 26 15:12:50 1998 > > Bit forwarding rate > > The number of bits per second of allowed traffic a firewall can be observed > > to transmit successfully to the correct destination interface in response > > to a specified offered load. > > > > Alternatively, you might try: > > The number of bits per second of traffic a firewall can be observed > to appropriately transmit to the correct destination interface(s) in response > to a specified offered load. > > This wording may help the move the burden from the user identifying classes of > traffic to a firewall discriminating what is to be forwarded - illegal, > blocked, malformed, wonderfully hunky-dory, or whatever. > > What constitutes appropriateness for the collection of this metric can be > defined in the companion methodology document. > Either definition works for me. Note that the terminology draft does define three classes of traffic: allowed, rejected, and (in the forthcoming version) illegal. I've received several inquiries in private e-mail about whether the bits in question constitute payload only or entire datagrams. My strong preference is that the latter be measured and I'd suggest adding language to that effect in the discussion of this definition. > > > > Connection > > A state in which nodes agree to exchange data using a known protocol. > > > This is wonderful as clarifying concept or term on which to build. As a > stand-alone metric, I'm not so sure . You referred to measuring connection > concurrency. This is something that I can and probably should measure wrt > firewalls. A connection overhead metric may yield better insight how an IUT > "lifts" or handles connection maintainance. > Kevin, I'd like a clarification of what's meant by "connection overhead." The memory cost of opening a socket is implementation-specific, and a DUT/SUT that uses 100 kbytes of memory for each connection may or may not move bits on the wire more slowly than one requiring only 10 kbytes. That's a firewalls internal issue, and probably not all that relevant to an end-to-end benchmark. Or did you mean something else here? Another option is to make the connection definition(s) very specific to the workings of TCP, UDP, and ICMP, and count only those conversations which have successfully navigated the setup procedures of each protocol. I'm reluctant to do that; better to have a more general definition. dn From owner-bmwg@BayNetworks.COM Fri Jun 26 22:45:35 1998 Received: from smtp-gw.BayNetworks.COM (ns5.baynetworks.com [194.133.90.101]) by dokka.maxware.no (8.8.5/8.8.5) with ESMTP id WAA03536 for ; Fri, 26 Jun 1998 22:45:33 +0200 Received: from mailhost.BayNetworks.COM ([132.245.135.115]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id WAA18229; Fri, 26 Jun 1998 22:28:15 +0200 (MET DST) Received: from pobox.engeast.BayNetworks.COM (pobox.engeast.baynetworks.com [192.32.61.6]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id QAA20377; Fri, 26 Jun 1998 16:28:12 -0400 (EDT) Received: (majordom@localhost) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) id QAA02704; Fri, 26 Jun 1998 16:22:44 -0400 for bmwg-outgoing Received: from mailhost.BayNetworks.COM (ns2.baynetworks.com [134.177.1.109]) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) with ESMTP id QAA02698; Fri, 26 Jun 1998 16:22:43 -0400 for Received: from smtp-gw.BayNetworks.COM (ext-ns3.baynetworks.com [192.32.253.3]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id NAA03733 for ; Fri, 26 Jun 1998 13:22:42 -0700 (PDT) Received: from lox.sandelman.ottawa.on.ca (lox.sandelman.ottawa.on.ca [209.151.24.2]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id QAA11366 for ; Fri, 26 Jun 1998 16:22:39 -0400 (EDT) Received: from istari.sandelman.ottawa.on.ca (istari.sandelman.ottawa.on.ca [209.151.24.30]) by lox.sandelman.ottawa.on.ca (8.8.7/8.8.8) with ESMTP id QAA26390 for ; Fri, 26 Jun 1998 16:22:33 -0400 (EDT) Received: from istari.sandelman.ottawa.on.ca ([[UNIX: localhost]]) by istari.sandelman.ottawa.on.ca (8.8.7/8.7.3) with ESMTP id QAA25548 for ; Fri, 26 Jun 1998 16:22:28 -0400 (EDT) Message-Id: <199806262022.QAA25548@istari.sandelman.ottawa.on.ca> To: bmwg@BayNetworks.COM Subject: Re: Thoughts on firewall performance draft In-reply-to: Your message of "Fri, 26 Jun 1998 15:25:00 EDT." <199806261925.PAA26657@pobox.engeast.BayNetworks.COM> Date: Fri, 26 Jun 1998 16:22:26 -0400 From: "Michael C. Richardson" Sender: owner-bmwg@BayNetworks.COM Precedence: bulk >>>>> "Kevin" == Kevin Dubray writes: >> This is wonderful as clarifying concept or term on which to build. As >> a stand-alone metric, I'm not so sure . You referred to measuring >> connection concurrency. This is something that I can and probably >> should measure wrt firewalls. A connection overhead metric may yield >> better insight how an IUT "lifts" or handles connection maintainance. Kevin> Kevin, I'd like a clarification of what's meant by "connection Kevin> overhead." The memory cost of opening a socket is Given n current connections, how much time does it take for the n+1'th connection to be opened. Kevin> implementation-specific, and a DUT/SUT that uses 100 kbytes of Kevin> memory for each connection may or may not move bits on the wire Kevin> more slowly than one requiring only 10 kbytes. That's a firewalls The internals issue is more to do with the kind of data structure that is used to store the connection state. Also, whether or not a new process needs to be created to handle that connection. Kevin> Another option is to make the connection definition(s) very Kevin> specific to the workings of TCP, UDP, and ICMP, and count only Kevin> those conversations which have successfully navigated the setup Kevin> procedures of each protocol. I'm reluctant to do that; better to Kevin> have a more general definition. ICMP isn't a protocol in the same way TCP and UDP are. TCP and UDP make use of IP services. Some of these services are carried by ICMP datagrams. A firewall that doesn't consider an ICMP MUST FRAGMENT to be associated with a TCP connection (i.e. Path MTU discovery) is violating the IP protocol. [The firewall needn't let it in: it could process it locally, adjusting its own view of fragment size and perhaps emitting an ICMP out the other end.] The only ICMP that may consistute a protocol *may* be ECHO/ECHO RESPONSE. :!mcr!: | Sandelman Software Works Corporation, Ottawa, ON Michael Richardson | SSH IPsec: http://www.ssh.fi/. Secure, strong, international Personal: mcr@sandelman.ottawa.on.ca. PGP key available. Corporate: sales@sandelman.ottawa.on.ca. From owner-bmwg@BayNetworks.COM Fri Jun 26 23:14:37 1998 Received: from smtp-gw.BayNetworks.COM (ns4.BayNetworks.COM [192.32.253.7]) by dokka.maxware.no (8.8.5/8.8.5) with ESMTP id XAA03843 for ; Fri, 26 Jun 1998 23:14:36 +0200 Received: from mailhost.BayNetworks.COM ([132.245.135.115]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id RAA14854; Fri, 26 Jun 1998 17:07:25 -0400 (EDT) Received: from pobox.engeast.BayNetworks.COM (pobox.engeast.baynetworks.com [192.32.61.6]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id RAA23713; Fri, 26 Jun 1998 17:06:52 -0400 (EDT) Received: (majordom@localhost) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) id RAA06779; Fri, 26 Jun 1998 17:02:22 -0400 for bmwg-outgoing Received: from mailhost.BayNetworks.COM (ns4.baynetworks.com [132.245.135.84]) by pobox.engeast.BayNetworks.COM (SMI-8.6/BNET-97/04/24-S) with ESMTP id RAA06772; Fri, 26 Jun 1998 17:02:21 -0400 for Received: from smtp-gw.BayNetworks.COM (ext-ns3.baynetworks.com [192.32.253.3]) by mailhost.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id RAA28638 for ; Fri, 26 Jun 1998 17:02:19 -0400 (EDT) Received: from copper.cmp.com (copper.cmp.com [192.155.65.19]) by smtp-gw.BayNetworks.COM (8.8.8/8.8.8) with ESMTP id RAA12476 for ; Fri, 26 Jun 1998 17:02:19 -0400 (EDT) Received: from NotesSMTP-01.cmp.com (gw57-25.cmp.com [192.155.57.25]) by copper.cmp.com (8.8.8/8.8.8) with SMTP id RAA07493 for ; Fri, 26 Jun 1998 17:01:02 -0400 (EDT) Received: from cmp.com ([192.155.84.32]) by NotesSMTP-01.cmp.com (Lotus SMTP MTA v4.6.1 (569.2 2-6-1998)) with SMTP id 8525662F.00736EB4; Fri, 26 Jun 1998 17:00:53 -0400 Message-ID: <35940C04.66CFD77F@cmp.com> Date: Fri, 26 Jun 1998 17:00:52 -0400 From: David Newman Reply-To: dnewman@cmp.com Organization: Data Communications magazine X-Mailer: Mozilla 4.05 [en] (WinNT; U) MIME-Version: 1.0 To: bmwg@BayNetworks.COM Subject: Re: Thoughts on firewall performance draft Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-bmwg@BayNetworks.COM Precedence: bulk > Kevin> Kevin, I'd like a clarification of what's meant by "connection > Kevin> overhead." The memory cost of opening a socket is > > Given n current connections, how much time does it take for the n+1'th > connection to be opened. > Great! That's a very good way of expressing connection setup time. But we have to define what's meant by connection in the first place. . .I'd be interested in your comments on the original proposal: A state in which nodes agree to exchange data using a known protocol. > Kevin> implementation-specific, and a DUT/SUT that uses 100 kbytes > of > Kevin> memory for each connection may or may not move bits on the > wire > Kevin> more slowly than one requiring only 10 kbytes. That's a > firewalls > > The internals issue is more to do with the kind of data structure > that > is used to store the connection state. Also, whether or not a new > process > needs to be created to handle that connection. Let's not get into internals; this is an effort to benchmark characteristics observed on the *outside* of the box. ;-) dn