Waji's Standards

Discussions on more advanced topics such as monolithic vs micro-kernels, transactional memory models, and paging vs segmentation should go here. Use this forum to expand and improve the wiki!
Post Reply
User avatar
Wajideu
Member
Member
Posts: 153
Joined: Wed Jul 30, 2014 1:05 am

Waji's Standards

Post by Wajideu »

I'm aware there are plenty of standards to choose from. I just don't like any of them. So of course, I'm going to bask in the irony of making my own.

/basking commences

One thing I happened to realize is the lack of organized criticism of any of the Unix standards despite the fact that anyone can look at them and find a mountain of things to complain about. I almost get the feeling there is some shadey group of 1337 linus worshipping hackers pounding away at their keyboards in their dank mother's basements. Insomniacs overdosed on WoW...Their skin, pale from the lack of nurishment from sunlight. Eyes burning, lights dim, monitors glaring off their glasses...[reflecting on personal experience] Where the hell was I going with this....

Anyhow, I'm going to post a few bs standards every now and then, and I want you guys to tell me what you think or any suggestions you may have. Don't be afraid to roast me, I'm a level 69 magus with oakflesh, I can take it.


Project Structure


Goals
  • The directory names should be human recognizable
  • The structure must be simple to navigate
  • The structure must be considerate of the planning process; with goals, rules, etc.

Presentation

Code: Select all

solution
 +- project1
 |   +- assets              project assets
 |   |   '- internal         '- internal assets
 |   +- locale              project localization
 |   |   +- messages         '- localized messages
 |   |   '- pointers         '- localized pointers
 |   +- outline             project outline
 |   +- output              project output
 |   |   '- <build>          '- project build
 |   +- scripts             scripts used for building the project
 |   |   +- passes           +- build passes
 |   |   '- tests            '- build tests
 |   +- shared              shared project data
 |   |   +- headers          +- code headers
 |   |   +- imports          +- code imports
 |   |   '- objects          '- code objects
 |   '- source              project source
 '- project2
     ...
Details

Projects Cannot Contain Other Projects
There are several reasons for this. Firstly, it just makes the directory structure easier to navigate. Secondly, this prevents the issue of "recursive make considered harmful". Thirdly, it makes the structure more compatable with ide's. The solution should not contain any directories aside from the project directories.

Assets and Resources
With C and C++ projects, there never really was a specific directory for resources that should be embedded into the object file itself. Some people just use 'res' or 'rsc'. Assets to be distributed with the application were usually stored in a 'share' directory. The naming scheme was borrowed from the Filesystem Hierarchy Standard. The problem with this is that almost everything in the FHS is either abbreviated or conjugated in some way, and the structure recursively follows the same pattern which while easy to remember also makes it easy to get lost. A better approach in my opinion would be to use a different directory structure to help differentiate it from the rest of the filesystem and just use symbolic links if backwards compatibility is desired.

Localization
While gettext is the defacto standard for localization, it's not very useful when it comes to some embedded systems (especially when the filesystem doesn't contain file names). So in addition to localizing messages, I figured it'd be a good idea to consider the localization of pointers as well. Obviously, localized pointers would be statically linked though unlike messages. Note that both the asset and locale directories contain the input files. Ie. /locale/messages would contain the .pot files for gettext. The output files of these 2 directories is placed in /output/<build>/assets' and '/output/<build>/locale'.

Project Outline
This directory is used for keeping track of information about the project such as the budget, goals, ideas, etc. Ie. Files like README, TODO, AUTHORS, ChangeLog, etc. from the usual C/C++ directory structure would go here.

Project Output
The output directory contains a different subdirectory for each build type. Ie. debug, release, etc.

Scripts
These are scripts used when building the project. They're particularly useful for unit tests or for projects that require multiple passes to build; like the gcc.

Headers and Imports
This was changed from 'include' and 'lib' to 'shared/headers' and 'shared/imports' for two reasons. First is to make a more clear distinction about what the directories are for. People have a bad habit of putting headers in the include directory that don't belong there. Second is to clear up vague terminology. The term 'library' can mean many things; static, shared, dynamic, runtime, etc. But the only types of files that should be in it are libraries used for linking. Additionally, in some programming languages like C#, the libraries contain metadata to use when compiling code rather than requiring the programmer to include headers.

Shared Objects
This directory was added out of consideration for static linking. Sometimes, it just isn't practical to create a static library. For example, you may only need reuse a single .o file, use a flat binary stub for something, or have multiple static libraries that use the same .o files.


Summary

I'm not claiming this is the holy grail or something, but I believe it's more elegant and easier to maintain than what is currently considered the 'standard'. My focus was to come up with something modular that would compliment the use of makefiles and shell scripts; but would be abstract enough to be useful for projects that go beyond simply programming.

Well, there you have it. Hit me peeps, my body is ready. [-o<
Last edited by Wajideu on Thu Oct 09, 2014 5:34 am, edited 1 time in total.
User avatar
b.zaar
Member
Member
Posts: 294
Joined: Wed May 21, 2008 4:33 am
Location: Mars MTC +6:00
Contact:

Re: Waji's Standards

Post by b.zaar »

Wajideu wrote:I'm aware there are plenty of standards to choose from.
That's right and they are all very useful or they wouldn't all exist. VBE 1, 1.2, 2, 3, VBE/AF... A little foresight on the fact you are trying to write an advanced device interface for an 8086 or a 80386+ in pmode might have help?

Also it takes many people to create a working standard like... UDI, which is widely used amongst the unix versions it was to unite and ease the development for hardware vendors to only release 1 driver.
Wajideu wrote:I almost get the feeling there is some shadey group of 1337 linus worshipping hackers pounding away at their keyboards in their dank mother's basements. Insomniacs overdosed on WoW...Their skin, pale from the lack of nurishment from sunlight. Eyes burning, lights dim, monitors glaring off their glasses...
Pity WoW wasn't ported to FreeBSD too... (maybe it was I don't play it)
Wajideu wrote:Anyhow, I'm going to post a few bs standards every now and then, and I want you guys to tell me what you think or any suggestions you may have.
Well it's not a standard until someone else uses it. And asking for an opinion on something that might be useful to someone else is how the UDI/EDI/CDI standard started right?
Wajideu wrote:Goals

The directory names should be human recognizable
The structure must be simple to navigate
The structure must be considerate of the planning process; with goals, rules, etc.
Well until my PC is programming itself I find this useful.
Wajideu wrote:Presentation

CODE: SELECT ALL
solution
+- project1
|   +- assets              project assets
|   |   '- internal         '- internal assets
|   +- locale              project localization
|   |   +- messages         '- localized messages
|   |   '- pointers         '- localized pointers
|   +- outline             project outline
|   +- output              project output
|   |   '- <build>          '- project build
|   +- scripts             scripts used for building the project
|   |   +- passes           +- build passes
|   |   '- tests            '- build tests
|   +- shared              shared project data
|   |   +- headers          +- code headers
|   |   +- imports          +- code imports
|   |   '- objects          '- code objects
|   '- source              project source
'- project2
     ...
Looks familar...
Wajideu wrote:Details...
I'm not that dedicated to this post
"God! Not Unix" - Richard Stallman

Website: venom Dev
OS project: venom OS
Hexadecimal Editor: hexed
User avatar
iansjack
Member
Member
Posts: 4685
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Waji's Standards

Post by iansjack »

One thing I happened to realize is the lack of organized criticism of any of the Unix standards despite the fact that anyone can look at them and find a mountain of things to complain about. I almost get the feeling there is some shadey group of 1337 linus worshipping hackers pounding away at their keyboards in their dank mother's basements. Insomniacs overdosed on WoW...Their skin, pale from the lack of nurishment from sunlight. Eyes burning, lights dim, monitors glaring off their glasses...Where the hell was I going with this....
Oh no, not another one who thinks it's cool to slag off Unix. It's so childish.

We get the message - you don't like Unix. Millions do - get used to it. The fact is that Unix is an immensely successful operating system. It's better to look at the reasons why than to just dismiss it.

I predict a poor future for your "standards".
User avatar
b.zaar
Member
Member
Posts: 294
Joined: Wed May 21, 2008 4:33 am
Location: Mars MTC +6:00
Contact:

Re: Waji's Standards

Post by b.zaar »

iansjack wrote:
One thing I happened to realize is the lack of organized criticism of any of the Unix standards despite the fact that anyone can look at them and find a mountain of things to complain about. I almost get the feeling there is some shadey group of 1337 linus worshipping hackers pounding away at their keyboards in their dank mother's basements. Insomniacs overdosed on WoW...Their skin, pale from the lack of nurishment from sunlight. Eyes burning, lights dim, monitors glaring off their glasses...Where the hell was I going with this....
Oh no, not another one who thinks it's cool to slag off Unix. It's so childish.

We get the message - you don't like Unix. Millions do - get used to it. The fact is that Unix is an immensely successful operating system. It's better to look at the reasons why than to just dismiss it.

I predict a poor future for your "standards".
How many versions of unix is enough?

I mean if the standards were right...

Unix has it's place... I don't hate unix
Last edited by b.zaar on Thu Oct 09, 2014 4:54 am, edited 1 time in total.
"God! Not Unix" - Richard Stallman

Website: venom Dev
OS project: venom OS
Hexadecimal Editor: hexed
User avatar
Wajideu
Member
Member
Posts: 153
Joined: Wed Jul 30, 2014 1:05 am

Re: Waji's Standards

Post by Wajideu »

b.zaar wrote:Also it takes many people to create a working standard like... UDI, which is widely used amongst the unix versions it was to unite and ease the development for hardware vendors to only release 1 driver.
That's exactly why I decided to ask for the opinions and suggestions of others. I'm ghetto-level OCD, so the structure of some GNU projects makes me want to rip my hair out. I would personally port as many of the GNU projects as possible to the new structure if people were happy with it; if only to easy my own butthurt.
b.zaar wrote:Well it's not a standard until someone else uses it. And asking for an opinion on something that might be useful to someone else is how the UDI/EDI/CDI standard started right?
Touche...Well, afaik, there isn't actually a standard for C/C++ project structure. The GNU coding standards don't cover it at least. Sometimes it's aggravating when I look at 2 projects that have the same directory structure, but the contents of said directories are completely different. Eg. 'conf' in one project might be the autotools configuration while in another it contains about 100 different machine specific makefiles.

iansjack wrote:Oh no, not another one who thinks it's cool to slag off Unix. It's so childish.

We get the message - you don't like Unix. Millions do - get used to it. The fact is that Unix is an immensely successful operating system. It's better to look at the reasons why than to just dismiss it.

I predict a poor future for your "standards".
lol, I have a love/hate relationship with Unix. Love the concept, hate the implementation. The entire reason I started this was because I was going to attempt to clean up the GNU/Linux codebase.
User avatar
iansjack
Member
Member
Posts: 4685
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Waji's Standards

Post by iansjack »

Wajideu wrote:The entire reason I started this was because I was going to attempt to clean up the GNU/Linux codebase.
Good luck with that modest ambition.
User avatar
Wajideu
Member
Member
Posts: 153
Joined: Wed Jul 30, 2014 1:05 am

Re: Waji's Standards

Post by Wajideu »

iansjack wrote:
Wajideu wrote:The entire reason I started this was because I was going to attempt to clean up the GNU/Linux codebase.
Good luck with that ambition.
I know I'll need it. I may has well have just exclaimed I'm going to paint my house with a toothbrush.

But...people say it needs done all the time and no one wants to take a stab at it.
User avatar
iansjack
Member
Member
Posts: 4685
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Waji's Standards

Post by iansjack »

I'd humbly suggest that starting your efforts with
I almost get the feeling there is some shadey group of 1337 linus worshipping hackers pounding away at their keyboards in their dank mother's basements. Insomniacs overdosed on WoW...Their skin, pale from the lack of nurishment from sunlight. Eyes burning, lights dim, monitors glaring off their glasses...Where the hell was I going with this....
is probably not the most effective way of persuading the Unix guys to your point of view. Or, indeed, of persuading anyone that it is worth taking the time to read the rest of your post.
User avatar
Wajideu
Member
Member
Posts: 153
Joined: Wed Jul 30, 2014 1:05 am

Re: Waji's Standards

Post by Wajideu »

iansjack wrote:I'd humbly suggest that starting your efforts with
I almost get the feeling there is some shadey group of 1337 linus worshipping hackers pounding away at their keyboards in their dank mother's basements. Insomniacs overdosed on WoW...Their skin, pale from the lack of nurishment from sunlight. Eyes burning, lights dim, monitors glaring off their glasses...Where the hell was I going with this....
is probably not the most effective way of persuading the Unix guys to your point of view. Or, indeed, of persuading anyone that it is worth taking the time to read the rest of your post.
The funny thing is that I was actually speaking from experience there. A property I lived at before had 2 houses on it. I stayed in the back house while my parents were in the front. The place was kind of rundown and smelled dingy. I had a bad habit of going on coffee benges and staying up for 2-3 days at a time playing runescape or working on projects.
User avatar
b.zaar
Member
Member
Posts: 294
Joined: Wed May 21, 2008 4:33 am
Location: Mars MTC +6:00
Contact:

Re: Waji's Standards

Post by b.zaar »

Wajideu wrote:
b.zaar wrote:Also it takes many people to create a working standard like... UDI, which is widely used amongst the unix versions it was to unite and ease the development for hardware vendors to only release 1 driver.
That's exactly why I decided to ask for the opinions and suggestions of others. I'm ghetto-level OCD, so the structure of some GNU projects makes me want to rip my hair out. I would personally port as many of the GNU projects as possible to the new structure if people were happy with it; if only to easy my own butthurt.
There are alternatives to the GNU lot, pdlibc would be one...

It's more a philosophy. GNU is great for Linux, they work together well. But a new OS doesn't need the same feel.

The GNU/Linux people want to stay with GNU and Linux... Fine, I'm not working towards pleasing them they are already totally happy... (oh wait)
Wajideu wrote:lol, I have a love/hate relationship with Unix. Love the concept, hate the implementation. The entire reason I started this was because I was going to attempt to clean up the GNU/Linux codebase.
That's a bigger dream that asking people to help shape some software. Do you know that Windows 10 is different from windows 8 because feedback (creating a standard humans like) was the motive?

What is this forum for?

Writing a better unix or writing something you created for more than 1 fuckin person?
"God! Not Unix" - Richard Stallman

Website: venom Dev
OS project: venom OS
Hexadecimal Editor: hexed
User avatar
Wajideu
Member
Member
Posts: 153
Joined: Wed Jul 30, 2014 1:05 am

Re: Waji's Standards

Post by Wajideu »

b.zaar wrote:What is this forum for?

Writing a better unix or writing something you created for more than 1 fuckin person?
I assumed both... Nearly all of the tutorials on this website depend on GNU/Linux packages somewhere. Either way you look at it, it would be greatly beneficial if some of the codebase could be cleaned up a bit.

But, anyhow, I'm not really that bent out of shape if you guys don't like this. I'm sitting on top of other ideas like figuring out a good way to wrap the FHS to reduce clutter. In the meantime, I'll going to take a stab at porting binutils to this structure to give you guys a physical example.
User avatar
Wajideu
Member
Member
Posts: 153
Joined: Wed Jul 30, 2014 1:05 am

Re: Waji's Standards

Post by Wajideu »

So, while I was porting binutils, I was considering the creation of an alternative to autotools called 'plan'. It would work similarly to 'make'; having a 'Planfile', but be used for configuring projects and generating Makefiles.

The syntax would be the same as make, but would have a few extra commands and a set of target prefixes which would allow you to specify plans with steps. Something along the lines of:

Code: Select all

CC			:= $(detect cc)


&MyFirstPlan:
	$(define HAS_PLAN)	# Define something


# Normal Makefile target here
somerandomtarget:
	@echo nothing to see here, move along...


&MySecondPlan: MyStep1 MyStep2
	$(assert $(CC),gcc)	# Require the c compiler to be called 'gcc'
	$(borrow MyFirstPlan)
^MyStep1:
	$(define DEBUG)		# Define something
	$(output "source/config.h")	# Output header
	@make all install
^MyStep2:
	$(undef DEBUG)		# Undefine something
	$(output "source/config.h")	# Output header
	@touch "source/error.c"
	@make all test


# Normal Makefile rule here
%.c:
	$(CC) -o $@ $<

Idk. I can't really come up with a good example atm, I'm running on fumes. It's basically just a Makefile template; where you 'plan' to 'make' something.
User avatar
b.zaar
Member
Member
Posts: 294
Joined: Wed May 21, 2008 4:33 am
Location: Mars MTC +6:00
Contact:

Re: Waji's Standards

Post by b.zaar »

Wajideu wrote:

Code: Select all

CC			:= $(detect cc)


&MyFirstPlan:
	$(define HAS_PLAN)	# Define something


# Normal Makefile target here
somerandomtarget:
	@echo nothing to see here, move along...


&MySecondPlan: MyStep1 MyStep2
	$(assert $(CC),gcc)	# Require the c compiler to be called 'gcc'
	$(borrow MyFirstPlan)
^MyStep1:
	$(define DEBUG)		# Define something
	$(output "source/config.h")	# Output header
	@make all install
^MyStep2:
	$(undef DEBUG)		# Undefine something
	$(output "source/config.h")	# Output header
	@touch "source/error.c"
	@make all test


# Normal Makefile rule here
%.c:
	$(CC) -o $@ $<
It looks like your plan builds a makefile, there's a thousand tools that do that kind of thing already. If make worked easily there wouldn't be a need for these. I don't have a tool to build C code, I write it myself.

All a make like build system needs is localized variables and then it would work how coders already think about code.

Code: Select all

#global
CC			:= cc


MyFirstPlan:
	$(define HAS_PLAN)	# Define something


# Normal Makefile target here
somerandomtarget:
	@echo nothing to see here, move along...


&MySecondPlan: MyStep1 MyStep2
	$(assert $(CC),gcc)	# Require the c compiler to be called 'gcc'
	$(borrow MyFirstPlan)

# this is a scripted rule so instructions like for and while can be used. Exec is a script command to escape to the shell.
MyStep1:{
	DEBUG = -D DEBUG=1		# local variable
#	@make all install           # recursive make again so what's the point of plan?
        
        #! run command        # a single line escape to shell - maybe not included if comments start with #

        # run multiple commands in the shell
        exec(
             run command      
             run commanu 2
        }
}

MyStep2:
#	$(undef DEBUG)		# this was never defined outside of MyStep1
	$(output "source/config.h")	# Output header
	@touch "source/error.c"
#	@make all test        # again recursion so make isn't [i]fixed[/i]


# Normal Makefile rule here
%.c:
	$(CC) $(CFLAGS) $(DEBUG) -o $@ $<
That's just my rough 2 seconds.
"God! Not Unix" - Richard Stallman

Website: venom Dev
OS project: venom OS
Hexadecimal Editor: hexed
gerryg400
Member
Member
Posts: 1801
Joined: Thu Mar 25, 2010 11:26 pm
Location: Melbourne, Australia

Re: Waji's Standards

Post by gerryg400 »

b.zaar wrote:All a make like build system needs is localized variables and then it would work how coders already think about code.
GnuMake has target specific variables. Is that what you mean ?
If a trainstation is where trains stop, what is a workstation ?
User avatar
b.zaar
Member
Member
Posts: 294
Joined: Wed May 21, 2008 4:33 am
Location: Mars MTC +6:00
Contact:

Re: Waji's Standards

Post by b.zaar »

gerryg400 wrote:
b.zaar wrote:All a make like build system needs is localized variables and then it would work how coders already think about code.
GnuMake has target specific variables. Is that what you mean ?
Sure but if you include a module or another project in your larger project then there can be conflicts. Also every project expects to be built seperately even if it's a library that does nothing on it's own.

I can't write this

Code: Select all


all: bintools pdlibc somethingelse

include pdlibc/Makefile

and expect it to play nice with the rest of my build process.

It's more about knowing the value of something at run time but letting the makefile know all the targets. $(objs) could be reused in every makefile without it interferring with a previous definition. It's about using encapsulation for each project.

Code: Select all


all: mod1 mod2

include mod1
include mod2

Code: Select all

mod1:{

    src := this that other
    objs := $(addprefix $(BUILDDIR),$(src))
   
    target1: $(objs)

}

Code: Select all

mod2:{

    src := file1 file2
    objs := $(addprefix $(BUILDDIR),$(src))
   
    target1: $(objs) target2 target3
    
    target2: {
       include mod3.mk  # all of the variables used inside of mod3.mk are now also private, could be handy to include original makefiles without modification
    }

    target3: {
         src := file3 file4
    }
}
The {} creates code blocks just as simply as an if, for or while block so they can be nested with private variables.

The internal target: dependencies table to make would then look something like

Code: Select all


all: mod1 mod2

mod1: #empty so must run

mod1.target1: mod1.$(objs)

mod2: #empty so must run

mod2.target1: mod2.$(objs) mod2.target2.whatever_is_included mod2.target3.$(objs)

"God! Not Unix" - Richard Stallman

Website: venom Dev
OS project: venom OS
Hexadecimal Editor: hexed
Post Reply