TL;DR: Stop what you are doing and go read both
Computer Architecture: a Quantatitive Approach and
Computer Organization and Design if you haven't do so already. Don't come back until to you have at least purchased or borrowed a copy of each. Trust me, this is something you want to do, and you won't regret it.
Your last post (save the request for further comments) indicated that you intend to replicate the 8086 ISA. I would argue that this is an astoundingly, blindingly horrible idea, as even Intel think the x86 architecture is trash. It was meant as a stop-gap when it was designed, and frankly, even when IBM picked it in 1980 there were better alternatives - the main reason they picked the
8088 in the first place was because they didn't want to spend for the 16-bit memory bus of the 8086 or the
Motorola 68000, but were told not to bother with an 8-bit system like the 8080 or the
6502. Since their whole goal was to 'embrace-extend-extinguish' the home computer market by squashing everyone else and then slowly turn them into smart terminals to connect to mainframes with.
This suited Intel down to their socks, as they wanted out of the home market and figured that IBM's evil scheme would net them just enough money to let them fix their
real CPU, the
iAPX 432.
Neither of these plans worked, obviously. The PC was something of a flop in the home market (compared to the mostly 8-bit home systems like the Apple II and the Commodore 64 anyway - they did do well in the small-business market) until Compaq came along and created the clone market, which had the effect of destroying IBM's smart terminal prospects and giving the home market a boost. Meanwhile, the 432 died of cancer of the capability mechanism, meaning that Intel had to double down on the 8086 or go out of business.
(
hardware capability-based security never really proved feasible with the then-current fabrication technology, as the silicon needed for it was bigger than that for the CPU itself, and interest in them died out before the die process reached a point where they might have been workable. Adding hardware support for garbage collection and object-oriented programming only compounded the problem. The design might work today, if anyone care to bother; in the late 1970s and early 1980s, though, it just wasn't possible.)
Anyway, back to the topic at hand. Unless you have a specific need for x86 compatibility, then I would strongly advise against it. In fact I would advise against it even if you did. You would be setting yourself up for frustration and failure if you trying to recreate that Frankenstein's monster of an ISA, no matter how widespread the actual Intel and licensed chips might be. And the performance will suck, too.
I won't direct you to a specific ISA architecture - and you should be glad
Geri is no longer here to demand that you to make a SUBLEQ OISC, that rabbit hole isn't worth going down except for funzies (it might be a fun project, but nothing more) - but I can definitely recommend looking at the classic
MIPS, the classic
Berkeley RISC, the
DLX (which is by the professors who designed the previous two; it is based on the MIPS but simplified even further), the
RISC-V, and maybe the
ARM for inspiration instead. Or conversely, something with a writable ISA like the Rekursiv, stack machines such as the
Lilith and
Ceres workstations, or a stack/register hybrid like the
Transputer. Maybe even take a look at the
Mill, though they have yet to make a working system and haven't released a lot of the details on it for legal reasons.
Or the 68000, though if you try to implement something like that in TTL, you'll end up with a mainframe - the 68k was comparable to the VAX in terms of complexity.
Note that I am saying, 'inspiration'. If you are going to do it from scratch, you might as well do your own design, right? Also, most of those are 32-bit designs, and while both the ARM and the MIPS have 16-bit variants If I were to suggest something to actual implement, it would be either the DLX (which is really really simple, open source, and completely documented) or the MIPS (which isn't as simple, but has usable ports of a lot of software), but even that isn't really something I would recommend.
I would, however, strongly recommend the books by the two main designers of the DLX, John Hennessy and David Patterson:
Computer Architecture: a Quantatitive Approach and
Computer Organization and Design. Seriously. They are great books, and absolutely essential for this topic - you would be doing yourself a disservice if you don't read them. note that the latter text has three versions, one focusing on MIPS, one on ARM, and one on RISC-V. While I am in agreement with Ivan Godard that, with modern chip fabrication, there's no real need for a classic RISC design in a commercial product (or a classic CISC, either), given the technology you mean to apply, a RISCy design makes tons of sense.