Page 1 of 1

I cannot get Stack Smash Protector working

Posted: Fri Aug 12, 2022 8:31 am
by tuanhtrng
Dear OSdevers,

I implemented SSP as the wiki instructed but failed to get it working. On release build, the stack overflow regress did not trigger the SSP. However, on the debug build, my whole regression test fails due to stack overflows! Why!?

I am learning RISC-V with OSdev. The OS is written in C++ using clang-devel from FreeBSD.

Re: I cannot get Stack Smash Protector working

Posted: Sun Aug 14, 2022 11:57 pm
by ccleaverr
Can you post some code? Including the code for the SSP and for the test.

How do your release and debug builds differ?

Re: I cannot get Stack Smash Protector working

Posted: Tue Aug 16, 2022 8:11 am
by tuanhtrng
I found the bug. It was a function from dlmalloc.

My kernel uses dlmalloc for dynamic memory.

Re: I cannot get Stack Smash Protector working

Posted: Tue Aug 16, 2022 2:23 pm
by kzinti
Are you saying that dlmalloc() has a bug? I find this surprising. Can you explain what was the problem with dlmalloc?