Page 1 of 4

Encouraging (Eventual) OS Adoption

Posted: Thu Nov 17, 2016 5:42 pm
by Schol-R-LEA
Brendan wrote:If pretty positive that, if my OS is ever usable/released, users will look back at the whining and criticism I get from various people (who won't even consider doing anything more than poor re-implementations of tired old ideas), and be completely unable to understand how I manage to resist the temptation to unleash the "wrathful purge" that this forum desperately requires in order to reverse its continuing de-evolution towards a "kindergarten idiocracy" (where those that have learnt not to think and to only regurgitate what they've seen elsewhere have somehow managed to decide they are "superior"; despite the fact that even the most inexperienced beginner has far more potential than they do, simply because an inexperience beginner hasn't been infected by the plague of "failure to consider anything different" yet).
Permit me to be blunt: you are talking as if you have never actually worked in professional programming. I assume that this is not true, given that you have been with this group for over a decade and clearly are quite knowledgeable about computer programming, but you give the impression of someone who thinks that computer programming is about designing and writing computer programs, someone who has never seen his special snowflakes get thrown into a big melting pile which some cat then urinates on (to use Peter Welch's oddly apt metaphor).

Nothing could be further from the truth. Those are the least important, and least difficult, parts of software development. The hard parts are all in interpersonal communication, and I have yet to see a mathematically solid algorithm or heuristic to solve that, never mind one to solve things like being hired by an idiot whose sole point of reference in IT is a fifteen-year-old article in Wired that he read while in the dentist's office, or to connect two 'standard-compliant' libraries which not only aren't compliant but are completely unusable, which the boss has mandated the use of come hell or high water.

I have seen companies hire two dozen programmers at rates of between $60K and $120K USD per annum and then keep them idle for six months waiting for programs to be fixed which never came (I was in the group for four of those months, and it was the easiest $20K I ever made, thank you, SIAC and Kaiser Permanente!).

I have seen a company president announce that no testing will be done on the company's software before it ships (and then, when I left, tell the graphic artist, my friend Sophianna, that she needed to learn Access 97 programming ASAP to cover for my absence).

I have seen managers fire programmers for telling the truth, and managers fire programmers for refusing to lie to customers.

I have seen a company president fire my friend Sophie (the same from the earlier story, who had been hired as a web artist and then dragooned into system admin) because she didn't back up a mission-critical drive which he'd refused to buy backup media for just one day prior, backup media which she had been asking him to authorize the purchase of for the previous six months.

I have seen a major bank put every one of their web developers through a $25K-per-head training course only to fire the whole team because they decided to hire offshore experts in a completely different system instead (a move which just happened to coincide with the dropping of the Dot-Bomb, but surely that had nothing to do with it of course...).

And oh yes, I have seen companies - and government offices, and NPOs - where every computer in the place was at least 12 years old, and where there were no prospects for replacing them in the near future.

These aren't edge cases. These are not flukes. And while some were short-lived startups that died of their endemic stupidities, others were big, successful corporations, or major government offices.

This is exactly how the majority of companies treat IT, because even at major software shops, programmers, IT techs, and system administrators are all generally seen as over-priced craftsmen whom they would replace as soon as something that looks cheaper comes along. The majority of companies, even companies whose primary products are software or IT services, view IT as overhead, not capital. Regardless of whether this is right or wrong, it is the truth.

Think I am wrong? If I were, things like The Daily WTF and the Tales From Tech Support subreddit wouldn't exist. The now defunct F***ed Company website wouldn't have existed. The Dot-Com bubble wouldn't have happened at all, because that was driven by people looking, not to make a product, but a quick buck, people who had no idea what 'the information superhighway' was but knew that it was raining venture capital thanks to it.

I would have expected you to have seen things at least as bad as these if you had been an IT professional for even a fraction of the time you have been on this forum. I can't see how you have kept up your belief that you can force people to accept your software solely on the basis of it being "better", according to you, unless you have decided to blame the cat for piddling on your snowflake, instead of either the guys who asked for snowflakes when they really needed soap flakes, or the ones who decided to throw them all into a big pile out under the summer sun.

Let me be even more blunt: QUALITY DOES NOT SELL SOFTWARE. Period. Marketing sells software. Microsoft, Oracle, IBM, Apple, Facebook, Google, and pretty much every other "successful" IT firm succeeded because they pounded people with the idea that they needed what they were selling. They didn't even have to know what it was, just that they needed it. Whether they did or not, or if it was any good or not, didn't matter. Most software is the electronic equivalent of a Pet Rock, yet a slick marketing campaign can convince many, many people that they can't live without some program whose purpose is a mystery to them even after they have tried to use it.

Yes, it is true that great marketing can fail if the product is terrible (though you'd be surprised how often that isn't the case). The converse is not true, however - high quality cannot sell a product if the marketing isn't there. I assume you know that part, but you keep speaking as if your system is destined to be successful, which seems to be as much a way of setting yourself up for failure as assuming it will fail is.

I truly do respect your intelligence, Brendan, I think you are probably a lot smarter than I am, and certainly more knowledgeable about PC system programming. I just can't help but think that you are overselling yourself based on that intelligence. The smartest people in the world can still fool themselves if the let their faith, confidence, or ego get the better of them. I am glad to see that you are doing something new and different, but the fact that you are denouncing others because they don't have the same high aspirations you do is a Problem, one that has and will continue to cause problems in this group if you don't address it.

Re: DMA Limitations

Posted: Thu Nov 17, 2016 11:00 pm
by Brendan
Hi,
Schol-R-LEA wrote:
Brendan wrote:If pretty positive that, if my OS is ever usable/released, users will look back at the whining and criticism I get from various people (who won't even consider doing anything more than poor re-implementations of tired old ideas), and be completely unable to understand how I manage to resist the temptation to unleash the "wrathful purge" that this forum desperately requires in order to reverse its continuing de-evolution towards a "kindergarten idiocracy" (where those that have learnt not to think and to only regurgitate what they've seen elsewhere have somehow managed to decide they are "superior"; despite the fact that even the most inexperienced beginner has far more potential than they do, simply because an inexperience beginner hasn't been infected by the plague of "failure to consider anything different" yet).
Permit me to be blunt: you are talking as if you have never actually worked in professional programming. I assume that this is not true, given that you have been with this group for over a decade and clearly are quite knowledgeable about computer programming, but you give the impression of someone who thinks that computer programming is about designing and writing computer programs, someone who has never seen his special snowflakes get thrown into a big melting pile which some cat then urinates on (to use Peter Welch's oddly apt metaphor).

Nothing could be further from the truth. Those are the least important, and least difficult, parts of software development. The hard parts are all in interpersonal communication, and I have yet to see a mathematically solid algorithm or heuristic to solve that, never mind one to solve things like being hired by an idiot whose sole point of reference in IT is a fifteen-year-old article in Wired that he read while in the dentist's office, or to connect two 'standard-compliant' libraries which not only aren't compliant but are completely unusable, which the boss has mandated the use of come hell or high water.

I have seen companies hire two dozen programmers at rates of between $60K and $120K USD per annum and then keep them idle for six months waiting for programs to be fixed which never came (I was in the group for four of those months, and it was the easiest $20K I ever made, thank you, SIAC and Kaiser Permanente!).

I have seen a company president announce that no testing will be done on the company's software before it ships (and then, when I left, tell the graphic artist, my friend Sophianna, that she needed to learn Access 97 programming ASAP to cover for my absence).

I have seen managers fire programmers for telling the truth, and managers fire programmers for refusing to lie to customers.

I have seen a company president fire my friend Sophie (the same from the earlier story, who had been hired as a web artist and then dragooned into system admin) because she didn't back up a mission-critical drive which he'd refused to buy backup media for just one day prior, backup media which she had been asking him to authorize the purchase of for the previous six months.

I have seen a major bank put every one of their web developers through a $25K-per-head training course only to fire the whole team because they decided to hire offshore experts in a completely different system instead (a move which just happened to coincide with the dropping of the Dot-Bomb, but surely that had nothing to do with it of course...).

And oh yes, I have seen companies - and government offices, and NPOs - where every computer in the place was at least 12 years old, and where there were no prospects for replacing them in the near future.

These aren't edge cases. These are not flukes. And while some were short-lived startups that died of their endemic stupidities, others were big, successful corporations, or major government offices.

This is exactly how the majority of companies treat IT, because even at major software shops, programmers, IT techs, and system administrators are all generally seen as over-priced craftsmen whom they would replace as soon as something that looks cheaper comes along. The majority of companies, even companies whose primary products are software or IT services, view IT as overhead, not capital. Regardless of whether this is right or wrong, it is the truth.

Think I am wrong? If I were, things like The Daily WTF and the Tales From Tech Support subreddit wouldn't exist. The now defunct F***ed Company website wouldn't have existed. The Dot-Com bubble wouldn't have happened at all, because that was driven by people looking, not to make a product, but a quick buck, people who had no idea what 'the information superhighway' was but knew that it was raining venture capital thanks to it.

I would have expected you to have seen things at least as bad as these if you had been an IT professional for even a fraction of the time you have been on this forum. I can't see how you have kept up your belief that you can force people to accept your software solely on the basis of it being "better", according to you, unless you have decided to blame the cat for piddling on your snowflake, instead of either the guys who asked for snowflakes when they really needed soap flakes, or the ones who decided to throw them all into a big pile out under the summer sun.
Did you forget that we're talking about OS development projects; where it's almost always "single designer/developer creates everything"? For this context, almost everything you've said above is completely irrelevant.
Schol-R-LEA wrote:Let me be even more blunt: QUALITY DOES NOT SELL SOFTWARE. Period. Marketing sells software. Microsoft, Oracle, IBM, Apple, Facebook, Google, and pretty much every other "successful" IT firm succeeded because they pounded people with the idea that they needed what they were selling.
Do you think that "same as everything you're already using; except less mature/stable/optimised, fewer drivers and no support" is an excellent way to convince people that they need what you're selling?

Have you considered that perhaps "does something that other OSs don't/can't support" is a much better way to convince people (that can benefit from whatever that feature is) that they need what you're selling because there is no alternative?


Cheers,

Brendan

Re: DMA Limitations

Posted: Fri Nov 18, 2016 12:23 am
by Schol-R-LEA
Brendan wrote:Did you forget that we're talking about OS development projects; where it's almost always "single designer/developer creates everything"? For this context, almost everything you've said above is completely irrelevant.
Only until it actually starts seeing use by others, and others begin contributing to the project. Or do you expect to write a fully functional, professional-quality system on your own? That hasn't been a reasonable proposition since the mid-1970s, and was only reasonable then because microcomputers were so limited in both hardware capacity and user expectations - in larger systems, one-man shows were a thing of the past by the time ITS was in the works, which was around 1966.
Brendan wrote:Do you think that "same as everything you're already using; except less mature/stable/optimised, fewer drivers and no support" is an excellent way to convince people that they need what you're selling?
I would normally say "of course not", except, that's exactly where Linux was until around 1998 or so, and where Linux desktop managers basically still are. Linux on the desktop may not be setting the world on fire, but it isn't something entirely negligible either.

Don't misunderstand me: I want you to do something new, I am thrilled to see you and others here trying new things. My concern is that there is a risk in trying a new direction, and you seem to have an awful lot of your ego riding on the success of this one. The problem is that you are berating people whose interest is in learning, for following a well-trod path, which - correct me if I am wrong here - is exactly what people do when learning something new for which an established path exists. Most of the people here don't have the same goals you do, or I do, and arguing that a hobby project needs to break new ground just for the sake of innovation is pretty condescending.
Brendan wrote:Have you considered that perhaps "does something that other OSs don't/can't support" is a much better way to convince people (that can benefit from whatever that feature is) that they need what you're selling because there is no alternative?
I am not trying to sell anything, and to the best of my knowledge neither are any of the other members except you, rdos (who has been at this longer than any of us, I think, and has actually sold his system in the past), and perhaps embryo and onlyonemac. Most of the posters here are indulging in a mix of curiosity and tinkering, not trying to develop a product. While I agree that this is not a reason to slack off - one should try to fulfill one's goals to the best of their ability, and success can pick the least predictable times to strike (e.g., we're stuck with a lot of BS because Dennis and Ken didn't think their game system would ever be used outside of Bell Labs) - it doesn't mean that one's reach must exceeds one's grasp when working on something they are trying to do for fun.

If your argument were directed towards me personally, I would see it very differently, because I do have ambitions for my work - ambitions of the sort that would lead to degrees and published papers rather than deliverable systems, but still - and, had you made this argument to me about my system, I would have deferred to your judgment (EDIT: sorry, that was a bit of a brainfart, I don't know why I originally used a word that meant the opposite of what I intended there). Thing is, this is a thread about a personal project meant to learn how something works. It isn't meant to see the light of day, and if it does it will be entirely by accident. It does not need to be held to the same standards you seek to hold yours to.

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 3:25 am
by rdos
The way the IT industry is working is the major reason why I wouldn't continue being in it unless I actually got paid for working on my OS. I'm certainly not learning anything about the new huge frameworks that big companies use to tie-up their customers. The wealth of hardware, standards, and secrecy about how hardware chips operate isn't encouraging. It's all part of tying people up so they have no time to develop better frameworks. It's big businesses that are setting the agenda, which include operating systems. You cannot create a successful operating system today unless you have the support of a huge number of hardware manufacturers that will write drivers for you. Bill Gates or Linus Torvalds would not have gotten their operating systems out today, and both of them were awful designs to begin with.

Re: DMA Limitations

Posted: Fri Nov 18, 2016 3:32 am
by Brendan
Hi,
Schol-R-LEA wrote:
Brendan wrote:Did you forget that we're talking about OS development projects; where it's almost always "single designer/developer creates everything"? For this context, almost everything you've said above is completely irrelevant.
Only until it actually starts seeing use by others, and others begin contributing to the project. Or do you expect to write a fully functional, professional-quality system on your own? That hasn't been a reasonable proposition since the mid-1970s, and was only reasonable then because microcomputers were so limited in both hardware capacity and user expectations - in larger systems, one-man shows were a thing of the past by the time ITS was in the works, which was around 1966.
The flaw in your thinking is that you don't distinguish between an OS (Windows, Unix, BeOS, DOS, AmigaOS, VMS) and an implementation of an OS (WindowsXP, Windows8, Reactos; Redhat, Ubuntu, Solaris, FreeBSD; Hiaku, BeOS R5.1; MS-DOS, DR-DOS, FreeDOS; AmigaOS 4.0, AROS, AtheOS; OpenVMS V8.4-2, FreeVMS).

I expect to create an OS (literally; a set of standards/specifications, etc) and some form of standardisation processes/committees to maintain it; and write a crude/minimal initial implementation of that OS (with very few drivers, apps, etc). By the time anyone is able to extend or improve the initial implementation, or create an alternative implementation, it will be far too late for anyone to significantly change the OS itself (the design).

An OS (the design), plus a crude/initial implementation of it, is something that can be done by a single person in about 2 to 10 years (depending on features, goals, etc). It is NOT hard to do, it's only hard to do it well.

Note that what matters is the quality of the OS (the design). The quality of the initial implementation isn't very important, as code/implementations are relatively trivial to replace (they're disposable).
Schol-R-LEA wrote:
Brendan wrote:Do you think that "same as everything you're already using; except less mature/stable/optimised, fewer drivers and no support" is an excellent way to convince people that they need what you're selling?
I would normally say "of course not", except, that's exactly where Linux was until around 1998 or so, and where Linux desktop managers basically still are. Linux on the desktop may not be setting the world on fire, but it isn't something entirely negligible either.
I'm fairly sure that the Linux marketing slogan is "When the only thing that matters is an application like Apache or mySQL, you can tolerate an OS that is worth nothing". ;)
Schol-R-LEA wrote:Don't misunderstand me: I want you to do something new, I am thrilled to see you and others here trying new things. My concern is that there is a risk in trying a new direction, and you seem to have an awful lot of your ego riding on the success of this one. The problem is that you are berating people whose interest is in learning, for following a well-trod path, which - correct me if I am wrong here - is exactly what people do when learning something new for which an established path exists. Most of the people here don't have the same goals you do, or I do, and arguing that a hobby project needs to break new ground just for the sake of innovation is pretty condescending.
I don't berate people who are trying to learn (ironically, I get criticised for not berating people that came here to learn, for unrelated reasons); I berate people that attempt to convince others that trying to do anything differently is a waste of time.

That fact is that every single thing that an "OS implementer" will ever implement is something that was once a "new and different" (and sometimes crazy sounding) invention that was created by someone who tried to go beyond the "status quo at the time".
Schol-R-LEA wrote:
Brendan wrote:Have you considered that perhaps "does something that other OSs don't/can't support" is a much better way to convince people (that can benefit from whatever that feature is) that they need what you're selling because there is no alternative?
I am not trying to sell anything, and to the best of my knowledge neither are any of the other members except you, rdos (who has been at this longer than any of us, I think, and has actually sold his system in the past), and perhaps embryo and onlyonemac. Most of the posters here are indulging in a mix of curiosity and tinkering, not trying to develop a product. While I agree that this is not a reason to slack off - one should try to fulfill one's goals to the best of their ability, and success can pick the least predictable times to strike (e.g., we're stuck with a lot of BS because Dennis and Ken didn't think their game system would ever be used outside of Bell Labs) - it doesn't mean that one's reach must exceeds one's grasp when working on something they are trying to do for fun.
Should people that are only indulging in a mix of learning, curiosity and tinkering be discouraged from learning about, being curious about, and tinkering with ideas that are not stale? Why would you (or anyone else) defend people that actively try to discourage the exploration of new ideas?
Schol-R-LEA wrote:If your argument were directed towards me personally, I would see it very differently, because I do have ambitions for my work - ambitions of the sort that would lead to degrees and published papers rather than deliverable systems, but still - and, had you made this argument to me about my system, I would have deferred to your judgment (EDIT: sorry, that was a bit of a brainfart, I don't know why I originally used a word that meant the opposite of what I intended there). Thing is, this is a thread about a personal project meant to learn how something works. It isn't meant to see the light of day, and if it does it will be entirely by accident. It does not need to be held to the same standards you seek to hold yours to.
Trust me - a lot of people secretly do have ambitions (it's the reason they're learning or curious or tinkering in the first place). The only difference is that most people don't feel like they can achieve their ambition (because of people like you that try to convince them it's unrealistic/impossible) and very few people are willing to be ridiculed (by people like you) for admitting they have ambitions. The difference between me and most people is that I stopped caring what people think a long time ago, and am not ashamed to admit I have ambitions.


Cheers,

Brendan

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 3:36 am
by Brendan
Hi,
rdos wrote:The way the IT industry is working is the major reason why I wouldn't continue being in it unless I actually got paid for working on my OS. I'm certainly not learning anything about the new huge frameworks that big companies use to tie-up their customers. The wealth of hardware, standards, and secrecy about how hardware chips operate isn't encouraging. It's all part of tying people up so they have no time to develop better frameworks.
You forgot the worst part - patents. :(
rdos wrote:It's big businesses that are setting the agenda, which include operating systems. You cannot create a successful operating system today unless you have the support of a huge number of hardware manufacturers that will write drivers for you. Bill Gates or Linus Torvalds would not have gotten their operating systems out today, and both of them were awful designs to begin with.
Yes; but in that case the goal becomes convincing hardware manufacturers to support the OS (rather than convincing end users). In the end it adds up to mostly the same thing - trying to do something that will convince someone.


Cheers,

Brendan

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 5:06 am
by rdos
Brendan wrote: Yes; but in that case the goal becomes convincing hardware manufacturers to support the OS (rather than convincing end users). In the end it adds up to mostly the same thing - trying to do something that will convince someone.
Yes, but that would turn me into a sales-person, not somebody designing and implementing an OS. I hate selling things, so that would make me uninterested in this endeavor. Besides, people that are good at selling their ideas often have bad designs and lousy products. Thus, I feel that if I'm successful at selling my design, the design would be lousy. Unless I have the backing of a large organization with sellers that are willing to listen to my ideas.

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 5:36 am
by Brendan
Hi,
rdos wrote:
Brendan wrote: Yes; but in that case the goal becomes convincing hardware manufacturers to support the OS (rather than convincing end users). In the end it adds up to mostly the same thing - trying to do something that will convince someone.
Yes, but that would turn me into a sales-person, not somebody designing and implementing an OS. I hate selling things, so that would make me uninterested in this endeavor. Besides, people that are good at selling their ideas often have bad designs and lousy products. Thus, I feel that if I'm successful at selling my design, the design would be lousy. Unless I have the backing of a large organization with sellers that are willing to listen to my ideas.
Yes; but in that case the goal becomes designing (and implementing an initial implementation of) an OS that's capable of selling itself. ;)


Cheers,

Brendan

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 1:42 pm
by tsdnz
Brendan wrote:.......Yes; but in that case the goal becomes designing (and implementing an initial implementation of) an OS that's capable of selling itself. ;)......
Very interesting discussions. I agree with Brendan. I am doing exactly that, I am designing an OS that will sell it's self. People will want it just because it is much faster, easier to program and does not come with all the dramas that existing OSes have. My design is not for desktops, it is for a cloud environment. I have been working on it for a couple of years, I was planning on developing the entire OS myself, but due to cancelled contracts, I have brought in an investor and we are looking at hiring to speed up development.

I truly believe that anything is possible, and that if we work as a team we could easily develop an amazing OS.

People are waiting for things in this world, they know the "system", they want their daily lives to be easier, I am going to provide such a solution, in the cloud OS space to start with.

Ali

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 2:49 pm
by matt11235
tsdnz wrote:cloud
Do you mean that it's highly available and scalable, or do you mean that it's just to be thrown on servers somewhere?

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 3:17 pm
by MichaelFarthing
tsdnz wrote: because it is much faster ... it is for a cloud environment.
Err...
Don't call us, we'll call you.

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 3:20 pm
by tsdnz
MichaelFarthing wrote:
tsdnz wrote: because it is much faster ... it is for a cloud environment.
Err...
Don't call us, we'll call you.
That's the one, you call me, if it is good enough, bring your wallet.

Ali

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 3:25 pm
by tsdnz
zenzizenzicube wrote:
tsdnz wrote:cloud
Do you mean that it's highly available and scalable, or do you mean that it's just to be thrown on servers somewhere?
Yes, highly available and scalable. It takes 3 servers just to start it.
It has a Distributor OS, an Account Controller OS, and a userspace OS, these are the minimum servers/OS required, there are others but they are for users to use and not necessary to start the system.

There are three OS types:
1. Kernel - This is just for ABIT, not available to users, everything runs at CPL 0, eg the distributor, account controller OSes...
2. Services - This is where high performance is available, basically the user get the entire server, all cores, etc. They control the NIC buffers etc, they are still sandboxed, eg, CPL 3. This is for single apps that require high performance, all cores, handle their own memory, etc...
3. userspace - This is just like a normal OS, you can send, receive, get memory, etc..., multiple apps for different users can run on this server core.

It can handle 40 million small packets (64 Bytes) per second in plus 40 million out, total of 80 million.
This is the distributor server, the one that connects the outside pipe to ABIT network.

The userspace servers can handle 3.5 million packets per server per core in (RX), it is very tricky to get the packets from the card to OS to app in userspace, with multiple users/apps.
I have set the maximum cores to 64 per server for userspace OS

Sending packets is easy, it can send 12 million per core.
I just did an example for a funding organisation, I showed them 29 million packets per second sent from usersapce, the % load on the machine was 40%.
I used windows as a test case, using the same C code, it managed 70,000. This can be increased to 300,000 easily.
Basically the c code was.

Code: Select all

set up UDP socket, etc..
while (true)
{
Send 4 bytes of udp
}
Each userspace server has 2 x 10 GB NICs available to the userspace.

I was testing the time it took to spin up app, I was very disappointed with my results, around 50,000 per second.
So I have changed the code, I have yet to do a full test but I should be around the 1 million plus easily per second.

Ali

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 4:14 pm
by Schol-R-LEA
tsdnz wrote: 1. Kernel - This is just for ABIT, not available to users,
Sorry, ABIT? I am blanking on the term, the only thing I know of called that was a mobo manufacturer that went out of business several years ago.
tsdnz wrote:It can handle 40 million small packets (64 Bytes) per second in plus 40 million out, total of 80 million.
Oh, really?

40,000,000 packets x 64 bytes x 8 bits = 20.48 Gbps, assuming that the 64 bytes includes the frames and no other traffic is occurring. Just how fat a pipe do you intend to feed to?

Similarly, if you are sending packets at 12M per core, then assuming that your server is running at a consistent 4.5 GHz for every core, that gives you roughly 375 cycles per packet, assuming full load, no cache misses, and no other CPU operations on that core including scheduling and the I/O operations themselves. This puts you (to zeroth approximation and with some slop for pipelining) in the range of 300-400 instructions available for the processing of each packet, maximum. Care to reconsider that claim?

The numbers aren't impossible ones, admittedly, but they seem rather improbable in the absence of more specific facts about the packet processing, packet contents and data sources, scheduling policy, I/O subsystems, and transmission speed of the networking hardware.

Re: Encouraging (Eventual) OS Adoption

Posted: Fri Nov 18, 2016 5:27 pm
by tsdnz
Schol-R-LEA wrote:Sorry, ABIT?
The name of all the OSes.
Schol-R-LEA wrote:40,000,000 packets x 64 bytes x 8 bits = 20.48 Gbps, assuming that the 64 bytes includes the frames and no other traffic is occurring. Just how fat a pipe do you intend to feed to?
40 GB per distributor OS, the server can easily handle this, the PCI & Memory do get hit hard, it is just like a load balancer, it is doing a massive amount of work, when in place I will not run it this hard.
Schol-R-LEA wrote:Similarly, if you are sending packets at 12M per core, then assuming that your server is running at a consistent 4.5 GHz for every core, that gives you roughly 375 cycles per packet, assuming full load, no cache misses, and no other CPU operations on that core including scheduling and the I/O operations themselves. This puts you (to zeroth approximation and with some slop for pipelining) in the range of 300-400 instructions available for the processing of each packet, maximum. Care to reconsider that claim?
1,900,000,000 (1.9GHz) / 12,000,000 = 158 Cycles, this is the server I have.
About 80 Cycles per Kernel call = 88 Cycles left, if sent one by one.
The OS can handle 256 packets per call, so only 1 Kernel call per 256 grouped packets, although I only used 64 for this test.
And yes it does it, also with only 1024 packet buffer per app.
The server has 2 x 10GB NIC's
Schol-R-LEA wrote:The numbers aren't impossible ones, admittedly, but they seem rather improbable in the absence of more specific facts about the packet processing, packet contents and data sources, scheduling policy, I/O subsystems, and transmission speed of the networking hardware.
It would be possible to have 100 Apps on a single core sending this many packets minus the small amount of context switching, still to test this, (12 Million / 100 = How much each app can do) , but if the Apps were doing processing this would have to be considered, for the test I had one App per core, tests were done using 1 core, 2 cores, etc...
Packet processing was not done, I just loaded a UDP packet with data, and sent it, then watched the Switch counters and Windows counters (Windows was the destination), windows could not handle much.

My issue is there is nothing that can prove ABIT.
I can send more than 40 million packets per second, I have reached 65 million, but I want it to handle in packets and out packets so I had to find the right balance.
The PCI and memory restricted how many NICs I could use.

I only have a limited number of 10GB NICs, 40 to be exact.
A lot are loaded into servers for testing.
I am willing to send a few to people who have the skills to test this OS some NICs and the OS binary.
But before I do this I need to see what they are capable of.

If you have Intel 82599 NICs I can send you the OS binary and you can see for yourself.
It will just send UDP "From Core xx, Msg [QWORD]", where [QWORD] is the message number as a QWORD, not text.

Ali