Interrupts with APIC, I/O APIC

[IanSeyler]

Is it possible to share the code? I have similar issues with the Keyboard (IRQ 1) and RTC (IRQ working in VirtualBox, QEMU, and Bochs but not VMware.

The Pure64 code can be seen here: http://code.google.com/p/pure64/source/ ... vn%2Ftrunk
Most of the APIC and I/O APIC code is in init_smp.asm

Best regards,
Ian Seyler

[Brendan]

Hi,

but when I set up IRQ 1 and enable interrupts nothing happens when I press any key on keyboard

Hmm. I have a sneaky suspicion - I might be completely wrong (it's just a guess), but...

In your keyboard initialisation code, insert a few dummy "in al,0x60" instructions. If that fixes all your problems, let me know and I'll explain why.


Cheers,

Brendan

[davispuh]

Is it possible to share the code? I have similar issues with the Keyboard (IRQ 1) and RTC (IRQ working in VirtualBox, QEMU, and Bochs but not VMware.

well, there isn't much to show... but I looked at your code and it's pretty good
there was some few suspicious things (or I just didn't understand that )
42-50 line, looks like you aren't specifying Spurious Vector, bits 0-3?

Code: Select all

; Step 2: Enable Local APIC on BSP
	mov rsi, [os_LocalAPICAddress]
	cmp rsi, 0x00000000
	je noMP				; Skip MP init if we didn't get a valid LAPIC address
	add rsi, 0xf0			; Offset to Spurious Interrupt Register
	mov rdi, rsi
	lodsd
	or eax, 0000000100000000b
	stosd

Hmm. I have a sneaky suspicion - I might be completely wrong (it's just a guess), but...

In your keyboard initialisation code, insert a few dummy "in al,0x60" instructions. If that fixes all your problems, let me know and I'll explain why.

I didn't saw any effect, also it's not just keyboard interrupt but I didn't got ANY interrupts in VirtualBox


yep, as I thought I had to configure local APIC now it works
(rdi - local APIC location)

Code: Select all

%define APIC.TPR 0080h
%define APIC.TPR.Mask 00ffh
%define APIC.LDR 00d0h
%define APIC.LDR.Mask (0ffh << 24)
%define APIC.DFR 00e0h
%define APIC.DFR.FlatModel (0fh << 28)
%define APIC.SVR 00f0h
%define APIC.SVR.Mask 00ffh
%define APIC.SVR.Enable (1 << 8)
%define APIC.ESR 0280h

mov eax, dword [rdi+APIC.DFR]                  ; Destination Format Register
or eax, APIC.DFR.FlatModel
mov dword [rdi+APIC.DFR], eax

mov eax, dword [rdi+APIC.LDR]                  ; Logical Destination Register
and eax, ~APIC.LDR.Mask
mov dword [rdi+APIC.LDR], eax

mov eax, dword [rdi+APIC.TPR]                  ; Task Priority Register
and eax, ~APIC.TPR.Mask
mov dword [rdi+APIC.TPR], eax 

mov eax, dword [rdi+APIC.SVR]                  ; Spurious Interrupt Vector Register
and eax, ~APIC.SVR.Mask
or eax, APIC.SVR.Enable | 248                    ; setting 248 interrupt as spurious 
mov dword [rdi+APIC.SVR], eax

xor eax, eax
mov dword [rdi+APIC.ESR], eax                  ; Error Status Register

btw looks like there's mistake in Intel manual 3A, Table 10-1 Local APIC Register Address Map
there ESR is marked as Read only, but later in 10.5.3 Error Handling

The ESR is a write/read register. Before attempt to read from the ESR, software
should first write to it. (The value written does not affect the values read subsequently;
only zero may be written in x2APIC mode.) This write clears any previously
logged errors and updates the ESR with any errors detected since the last write to the
ESR.

[M2004]

Here's my example of redirecting kbd IRQ through lapic instead of PIC. Requires
long mode and IDT set up properly. Also PIC needs to be shut down beforehand.

Code: Select all

use64	 
;Constants.
;----------
	TASK_PRIORITY_REGISTER			=0080H 
	DESTINATION_FORMAT_REGISTER		=00E0H
	SPURIOUS_INT_VEC_REG			=0xF0			

	LVT_TIMER_REGISTER			=0320H 				
	LVT_PERFORMANCE_MONITORING_COUNTERS	=0340H 				
	
	LVT_LINT0_REGISTER			=0350H 				
	LVT_LINT1_REGISTER			=0360H 				
	LVT_ERROR_REGISTER			=0370H 


	SPURIOUS_INT_VECTOR			=0x4f	;use vector 0x4f for the spurious int.

;.........

	;Setup long mode and IDT......
	;------------------------------
	;Handle LAPIC and IOAPIC setup.
	;------------------------------
	mov eax,0xfee00000
	mov dword[Local_Apic_address],eax	 	;Presume lapic base address and IOapic base address.
							;Note: Correct way of finding the base 
							;address's is through apic /mp spec table parsing.
	mov eax,0xfec00000
	mov dword[ACPI_IOAPIC_Base_Address_Table],eax 	;Presumme lapic base address and IOapic base address.

	call LM_Enable_Local_Apic			;Try to enable acpi.

	call LM_Disable_All_Lapic_IRQs			;Make sure that all IOapic ints are off.
	call LM_Initialize_Local_APIC_Irqs		;Setup Local apic to accept interruptions.

	;Let's enable keyboard interrupt (irq1) and direct it to BSP
	;-------------------------------------------------------------
	mov r9b,0					;r11b-->Requested IOAPIC redirection table entry to be enabled.
	mov r10b,0x21					;r10b-->Requested interrupt vector 
	mov r11b,1					;r9b -->APIC ID number where the interrupt will be redirected.
	call LM_Enable_IOAPIC_IRQ 			;Enable APIC KBD int

	;Continue with your code here.....
;.....

;Variables
;----------
align 8
	ACPI_IOAPIC_Temp_redir_entry		dq 0
	Local_Apic_address			dq 0	
	ACPI_IOAPIC_Base_Address_Table		dq 0,0,0,0

;****************************************************************************************************
;LM_Enable_Local_Apic	 		The procedure tries to enable the local apic.
;
;
;
;  Input:  --	
;
;  Output: --
;
;***********************************************************************************************
align 8

LM_Enable_Local_Apic:
	push rax
	push rdi

	cmp qword[Local_Apic_address],0		;Prevent illegal access.
	jz .no_apic_adress

	;Read current data, change needed bit(s)
	;and write the data back.
	;-----------------------------------------
	
	mov rdi,SPURIOUS_INT_VEC_REG		;Read Spurious Interrupt Vector Register
	call LM_Read_Apic_Register_Dword
	
	or eax, 0000000100000000b		;Set bit8		
	
	mov rdi,SPURIOUS_INT_VEC_REG		;Write Spurious Interrupt Vector Register
	call LM_Write_Apic_Register_Dword

.no_apic_adress:
	pop rdi
	pop rax
        ret					;The end of procedure

 ;*************************************************************************************************************
 ;LM_Disable_All_Lapic_IRQs	The procedure disables all lapic interrupts.
 ;
 ;  Input:  --	
 ;
 ;  Output: --
 ;
 ;*************************************************************************************************************
 align 8
 
 LM_Disable_All_Lapic_IRQs:
	push rax rbx rcx rdx rdi rsi r8 r9 r10 r11
 
 	mov  r10,24					;Handle 24 table entries.
  	mov  r8,0x10					;Start from 1st IO redirection table register.
 
 .read_next_entry:
 	mov qword[ACPI_IOAPIC_Temp_redir_entry],0	;Clear the variable.
 
 	;1) Read next IO redirection table entry from IOapic.
 	;-----------------------------------------------------
 	mov rbx,r8					;rbx--> Read the lower dword of IO redirection table register 
 	mov rcx,0					;rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)
 	call LM_Read_IOapic_Register
 	mov dword[ACPI_IOAPIC_Temp_redir_entry],eax
 
 	mov rbx,r8					;rbx--> Read the higher dword of IO redirection table register 
 	add rbx,1					;Read upper dword.
 	mov rcx,0					;rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)
 	call LM_Read_IOapic_Register
 	mov dword[ACPI_IOAPIC_Temp_redir_entry+4],eax
  
 	;2) Check whether the interrupt mask is allready on.
 	;----------------------------------------------------
 	bt qword[ACPI_IOAPIC_Temp_redir_entry],16	;Test bit16.  Interrupt mask. 
 	jc .prepare_to_read_next_entry			;If bit16 is set to 1, the mask is allreaady on and there's no need
 							;to tweak it.
  	bts qword[ACPI_IOAPIC_Temp_redir_entry],16	;Set Interrupt mask. 

 	;3) The interrupt mask wasn't set, so we need to 
 	;write the tweaked table entry back to IOAPIc.
 	;---------------------------------------------------
 	mov eax,dword[ACPI_IOAPIC_Temp_redir_entry]
 	mov rbx,r8					;rbx--> Write the lower dword of IO redirection table register 
 	mov rcx,0					;rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)
 	call LM_Write_IOapic_Register
 
 	mov eax,dword[ACPI_IOAPIC_Temp_redir_entry+4]
 	mov rbx,r8					;rbx--> Write the higher dword of IO redirection table register 
 	add rbx,1					;Read upper dword.
 	mov rcx,0					;rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)
 	call LM_Write_IOapic_Register
 
 .prepare_to_read_next_entry:
 	add r8,2					;Read next entry from it's low dword. 
 	dec r10
 	jnz .read_next_entry
 
 	
 .done:
	pop r11 r10 r9 r8 rsi rdi rdx rcx rbx rax
	ret						;The end of procedure

;*************************************************************************************************************
;LM_Initialize_Local_APIC_Irqs	The procedure prepares the local apic controller to accept interrputs. 
;
;
;
;  Input:  --	
;
;  Output: --
;
;*************************************************************************************************************
align 8

LM_Initialize_Local_APIC_Irqs:
	push rax rbx rcx rdx rdi rsi r8 r9 r10 r11


	cmp qword[Local_Apic_address],0			;Prevent illegal access.
	jz .done

	;1) Set task priority (baseaddress +00080h)
	;--------------------------------------------
	mov rdi,TASK_PRIORITY_REGISTER			;Read current data.
	call LM_Read_Apic_Register_Dword
	mov al,0					;Clear Task Priority (bits 7:4) and Task Priority Sub-Class (bits 3:0)

	mov rdi,TASK_PRIORITY_REGISTER			;Perform the write.
	call LM_Write_Apic_Register_Dword

	;2) Set timer interrupt vector (baseaddress +00320h)
	;-----------------------------------------------------
	mov rdi,LVT_TIMER_REGISTER			;Read current data.
	call LM_Read_Apic_Register_Dword

	bts eax,16					;bit16:Mask interrupts (0==Unmasked, 1== Masked)
	mov rdi,LVT_TIMER_REGISTER			;Perform the write.
	call LM_Write_Apic_Register_Dword

	;3) Set performance counter vector (baseaddress +00340h)
	;Performance may or may not exist.Intel 3a-3b states that
	;offset 00340h may vary as well.
	;--------------------------------------------------------
;	mov rdi,LVT_PERFORMANCE_MONITORING_COUNTERS	;Read current data.
;	call LM_Read_Apic_Register_Dword
;
;	bts eax,16					;bit16:Mask interrupts (0==Unmasked, 1== Masked)
;;	mov eax, 0x10000				;0x10000 (10000000000000000b) to disable performance counter interrupts		
;	mov rdi,LVT_PERFORMANCE_MONITORING_COUNTERS	;Perform the write.
;	call LM_Write_Apic_Register_Dword

	;4) Set local interrupt 0 (baseaddress +00350h)
	;--------------------------------------------------------
	mov rdi,LVT_LINT0_REGISTER			;Read current data.
	call LM_Read_Apic_Register_Dword
	mov al, 0					;Set interrupt vector (bits 7:0)
	bts eax,8					;Delivery Mode (111b==ExtlNT] (bits 10:8)
	bts eax,9					
	bts eax,10

	bts eax,15					;bit15:Set trigger mode to Level (0== Edge, 1== Level)
							;This flag (Trigger mode) is only used when the delivery mode is 
							;Fixed. When the delivery mode is  NMI, SMI, or INIT, the trigger
							;mode is always edge sensitive.When the delivery mode is ExtINT, 
							;the trigger mode is always level sensitive. The timer and error 
							;interrupts are always treated  as edge sensitive. 	
	btr eax,16					;bit16:unmask interrupts (0==Unmasked, 1== Masked)

	mov rdi,LVT_LINT0_REGISTER			;Perform the write.
	call LM_Write_Apic_Register_Dword

	;5) Set local interrupt 1 (baseaddress +00360h)
	;--------------------------------------------------------
	mov rdi,LVT_LINT1_REGISTER			;Read current data.
	call LM_Read_Apic_Register_Dword

	mov al, 0 					;Set interrupt vector (bits 7:0)
	btr eax,8					;Delivery Mode (000b==Fixed] (bits 10:8)
	btr eax,9					
	btr eax,10

	btr eax,15					;bit15:Set trigger mode to Edge (0== Edge, 1== Level)
							;This flag (Trigger mode) is only used when the delivery mode is 
							;Fixed. When the delivery mode is  NMI, SMI, or INIT, the trigger
							;mode is always edge sensitive.When the delivery mode is ExtINT, 
							;the trigger mode is always level sensitive. The timer and error 
							;interrupts are always treated  as edge sensitive. 	
	btr eax,16					;bit16:unmask interrupts (0==Unmasked, 1== Masked)

	mov rdi,LVT_LINT1_REGISTER			;Perform the write.
	call LM_Write_Apic_Register_Dword
	
	;6) Set error interrupt [baseaddress +00370h)
	;--------------------------------------------------------
	mov rdi,LVT_ERROR_REGISTER			;Read current data.
	call LM_Read_Apic_Register_Dword

	mov al, 0	;LVT_LINT_ERR_VECTOR		;Clear interrupt vector (bits 7:0)
	bts eax,16					;bit16:Mask interrupts (0==Unmasked, 1== Masked)

	mov rdi,LVT_ERROR_REGISTER			;Perform the write.
	call LM_Write_Apic_Register_Dword

	;7) Setup the "destination format register"
	;--------------------------------------------------------
	mov rdi,DESTINATION_FORMAT_REGISTER		;Read current data.
	call LM_Read_Apic_Register_Dword

	bts eax,28					;Flat Model is selected by programming DFR bits 28 through 31 
	bts eax,29					;to 1111.
	bts eax,30
	bts eax,31

	mov rdi,DESTINATION_FORMAT_REGISTER		;Perform the write.
	call LM_Write_Apic_Register_Dword
	
	;8) Set spurious interrupt [baseaddress +00F0h)
	;--------------------------------------------------------

	mov rdi,SPURIOUS_INT_VEC_REG			;Read current data.
	call LM_Read_Apic_Register_Dword
	
	mov al,SPURIOUS_INT_VECTOR			;Set interrupt vector (bits 7:0)
	bts eax,8					;bit8: APIC Software Enable/Disable, (0==APIC Disabled,1==APIC Enabled)
	
	;Intel 3a-3b manual says that bit12 (EOI-Broadcast Suppression)
	;and bit9 (Focus Processor Checking ) operations are not 
	;supported by all configurations. Let's disable them.
	;--------------------------------------------------------------
	
	bts eax,12					;bit12: EOI-Broadcast Suppression (0==Enabled, 1== Disabled)
	bts eax,9					;bit9: Focus Processor Checking (0==Enabled 1==Disabled)
	
	mov rdi,SPURIOUS_INT_VEC_REG			;Perform the write.
	call LM_Write_Apic_Register_Dword

.done:
	pop r11 r10 r9 r8 rsi rdi rdx rcx rbx rax
        ret						;The end of procedure

;*************************************************************************************************************
;LM_Enable_IOAPIC_IRQ		The procedure enables the requested IOAPIC IRQ from the IOAPIC redirection
;				table entry[x]. 
;
;
;
;  Input:  r11b-->Requested IOAPIC redirection table entry to be enabled..
;	   r10b-->Requested interrupt vector.
;	   r9b -->APIC ID number where the interrupt will be redirected.
;
;  Output: --
;
;*************************************************************************************************************
align 8

LM_Enable_IOAPIC_IRQ:
	push rax rbx rcx rdx rdi rsi r8 r9 r10 r11
	pushfq
 
	and r9,0xff					;Clear unwanted bits. 
	and r10,0xff					;Clear unwanted bits. 
	and r11,0xff					;Clear unwanted bits. 

	shl r11,1					;Entry number*2
	mov r8,0x10					;Get first table entry.
	add r8,r11					;Point to requested table entry.
 
 .read_next_entry:
 	mov qword[ACPI_IOAPIC_Temp_redir_entry],0	;Reset the variable
 
 	;1) Read next IO redirection table entry from IOapic.
 	;-----------------------------------------------------
 	mov rbx,r8					;rbx--> Read the lower dword of IO redirection table register 
 	mov rcx,0					;rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)
 	call LM_Read_IOapic_Register
 	mov dword[ACPI_IOAPIC_Temp_redir_entry],eax
 
 	mov rbx,r8					;rbx--> Read the higher dword of IO redirection table register 
 	add rbx,1					;Read upper dword.
 	mov rcx,0					;rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)
 	call LM_Read_IOapic_Register
 	mov dword[ACPI_IOAPIC_Temp_redir_entry+4],eax
 
  	;2) Adjust the bits for low dword
 	;----------------------------------------------------
 	mov byte[ACPI_IOAPIC_Temp_redir_entry],r10b	;Set the interrupt vector.(bits7:0)
 
  	btr qword[ACPI_IOAPIC_Temp_redir_entry],8	;Set delivery mode (bits10:8) to (000b==fixed) 
  	btr qword[ACPI_IOAPIC_Temp_redir_entry],9
  	btr qword[ACPI_IOAPIC_Temp_redir_entry],10

  	btr qword[ACPI_IOAPIC_Temp_redir_entry],11	;Set the Destination mode to physical (=0).(bit11)

  	btr qword[ACPI_IOAPIC_Temp_redir_entry],13	;Set int PIN polarity. (Active high (=0) for ISA int's)(bit13)
  	btr qword[ACPI_IOAPIC_Temp_redir_entry],15	;Set the trigger mode (Edge trigger (=0) for ISA int's.(bit15)

  	btr qword[ACPI_IOAPIC_Temp_redir_entry],16	;Clear the interrupt mask.(bit16)

  	;3) Adjust the bits for high dword
 	;----------------------------------------------------

 	mov byte[ACPI_IOAPIC_Temp_redir_entry+7],r9b	;Set the requested APIC ID.


 	;4) Write the tweaked table entry back to IOAPIc.
 	;---------------------------------------------------
 	mov eax,dword[ACPI_IOAPIC_Temp_redir_entry]
 	mov rbx,r8					;rbx--> Write the lower dword of IO redirection table register 
 	mov rcx,0					;rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)
 	call LM_Write_IOapic_Register
 
 	mov eax,dword[ACPI_IOAPIC_Temp_redir_entry+4]
 	mov rbx,r8					;rbx--> Write the higher dword of IO redirection table register 
 	add rbx,1					;Read upper dword.
 	mov rcx,0					;rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)
 	call LM_Write_IOapic_Register

.done:
	popfq
	pop r11 r10 r9 r8 rsi rdi rdx rcx rbx rax
        ret						;The end of procedure

;****************************************************************************************************
;LM_Read_Apic_Register_Dword 		The procedure reads a dword from the requested Local APIC 
;					register.
;
;  Input:  rdi--> Register offset	
;
;  Output: eax -->Read register contents.
;
;***********************************************************************************************
align 8

LM_Read_Apic_Register_Dword:
	push rsi

	cmp qword[Local_Apic_address],0		;Prevent illegal access.
	jz .no_apic_adress

	mov rsi,qword[Local_Apic_address]	;Get local apic address.
	add rsi, rdi				;Add the offset  
	mov eax, dword[rsi]			;Read the dword.

.no_apic_adress:

	pop rsi
        ret					;The end of procedure

;****************************************************************************************************
;LM_Write_Apic_Register_Dword 		The procedure writes the contents o feax into the requested
;					Local  APIC register.
;
;  Input:  rdi--> Register offset
;	   eax--> dword to be written.
;
;  Output: --
;
;****************************************************************************************************
align 8

LM_Write_Apic_Register_Dword:
	push rsi

	cmp qword[Local_Apic_address],0		;Prevent illegal access.
	jz .no_apic_adress

	mov rsi,qword[Local_Apic_address]	;Get local apic address.
	add rsi, rdi				;Add the offset  
	mov dword[rsi],eax			;Write the dword.

.no_apic_adress:

	pop rsi
        ret					;The end of procedure

 ;************************************************************************************************************
 ;LM_Select_IOAPIC_Register_Address	 	The procedure selects the requested IOapic register.
 ;
 ;
 ;
 ;  Input:  rbx--> IOapic register offset to be selected.(IOREGSEL)
  ;
 ;  Output: --
 ;
 ;************************************************************************************************************
 align 8
 
 LM_Select_IOAPIC_Register_Address:
	push rax rbx rcx rdx rdi rsi r8 r9 r10

	;Select requested IOWIN register through IOREGSEL.
	;--------------------------------------------------
	and rbx,0xff

	mov rsi,qword[ACPI_IOAPIC_Base_Address_Table]
	mov r10,rsi					;Duplicate the address.
	add rsi,0x0
	lodsd						;Read IOREGSEL register

	mov al,bl					;Modify bit7:0  (APIC register Address)
	mov eax, eax					;Write offset
	mov rdi,r10					;Get the base address.
	add rdi,0x0
	stosd						;Write IOREGSEL register

 .done:
	pop r10 r9 r8 rsi rdi rdx rcx rbx rax
	ret						;The end of procedure

;****************************************************************************************************
;LM_Write_IOapic_Register	 		The procedure writes to requested IOapic register.
;
;
;
;  Input: 
;	   eax--> Value to be written. (IOWIN)
;	   rbx--> IOapic register offset to be written.(IOREGSEL)
;	   rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)	
;
;  Output: --
;
;***********************************************************************************************
align 8

LM_Write_IOapic_Register:
	push rax rbx rcx rdx rdi rsi r8 r9 r10
							;rbx--> IOapic register offset to be selected.(IOREGSEL)	
	call LM_Select_IOAPIC_Register_Address

	mov eax, eax					;Write offset
	mov rdi,qword[ACPI_IOAPIC_Base_Address_Table]	;Get the base address.
	add rdi,0x10
	stosd						;Write IOWIN register

.done:
	pop r10 r9 r8 rsi rdi rdx rcx rbx rax
        ret						;The end of procedure

;****************************************************************************************************
;LM_Read_IOapic_Register	 		The procedure reads requested IOapic register.
;
;  Input: 
;	   rbx--> IOapic register offset to be read.(IOREGSEL)
;	   rcx--> IOapic controller number to be accessed, (0==1st, 1==2nd and so on...)	
;
;  Output: eax--> Read value from IOWIN register.
;
;***********************************************************************************************
align 8
LM_Read_IOapic_Register:
	push rbx rcx rdx rdi rsi 
							;rbx--> IOapic register offset to be selected.(IOREGSEL)	
	call LM_Select_IOAPIC_Register_Address

	mov rsi,qword[ACPI_IOAPIC_Base_Address_Table]
	add rsi,0x10
	lodsd						;Read IOWIN register

.done:
	pop rsi rdi rdx rcx rbx 
	ret						;The end of procedure

Regards
Mac2004

[IanSeyler]

I reactivated the legacy timer (IRQ 0) for testing and it is triggered under VMware (therefore I would say the IO APIC is working correctly). So it looks like VMware doesn't like the RTC setup but it works fine in QEMU, Bochs, and VirtualBox. Any ideas? The RTC in VMware worked fine using the PIC.

[M2004]

So it looks like VMware doesn't like the RTC setup but it works fine in QEMU, Bochs, and VirtualBox. Any ideas? The RTC in VMware worked fine using the PIC.

Perhaps WMware has got a buggy apic implementation?
Have you tested on real machines?

regards
Mac2004

[Nable]

Oh, may be my question is possible here.
Under hypervisor (yes, it's already mentioned here Palacios with many patches) linux kernel behaves very strange:

Code: Select all

[    0.128094] Using local APIC timer interrupts.
[    0.128095] calibrating APIC timer ...
[    1.252468] ... lapic delta = 183287420
[    1.252659] ... PM-Timer delta = 0
[    1.252849] ..... delta 183287420
[    1.253038] ..... mult: 7872134746
[    1.253228] ..... calibration result: 293259872
[    1.253420] ..... CPU clock speed is 29325.9873 MHz.
[    1.253613] ..... host bus clock speed is 29325.9872 MHz.
[    1.253805] ... verify APIC timer
[    2.551835] ... jiffies delta = 10
[    2.552024] ... jiffies result ok

This is a part from dmesg for kernel-2.6.32-71.29.1.el6 (CentOS 6). It boots, but time is slowed down (you can call `date' more and more and see that time ticks about 10 times slower than it should).
But kernel-2.6.18-274.7.1.el5 (CentOS 5) worked good..
Legacy timer 8042 is not available and even not connected in mptable and acpi.

Maybe anyone can tell in a few words what has changed in calibration routine or anyone knows good online service for comparing and diffing such amounts of code. Any help is appreciated.

[IanSeyler]

I just tested on real hardware and got the same results as on VMware. So it looks like there is a problem with my code. What a pain that it works fine in the other three emulators though. The timer seems to be firing correctly. I will add some debugging code that enables the keyboard to see if I get interrupts from it.

[IanSeyler]

I added a keyboard handler and it works as expected in QEMU, Bochs, VirtualBox, and more importantly VMware! So I'm pretty sure at this point that the I/O APIC is working correctly. There must be a bug somewhere with the RTC handling.

Do I still need to handle the cascade IRQ (2) when using the I/O APIC? I would assume not.
Is it possible that the RTC is being mapped to a different interrupt number (like the legacy timer is moved from 0 to 2) and I am missing it?

I know that the RTC does not fire at all since if it only fired once I would still see some information on the screen.

Any ideas would be greatly appreciated. I could use the legacy Timer for the CPU speed calculations as well as the SMP init but I like the RTC better (Since I can set it to fire at exactly 1024Hz).

-Ian

[M2004]

Is it possible that the RTC is being mapped to a different interrupt number (like the legacy timer is moved from 0 to 2) and I am missing it?

In that case you need an interrupt source override structure (ACPI) stating that.

Regards
Mac2004

[IanSeyler]

In that case you need an interrupt source override structure (ACPI) stating that.

I checked and there were none. Just though maybe it was an unwritten rule

How about the I/O APIC entries? I read (earlier in this thread I think) that ISA and PCI interrupts are triggered differently. When the I/O APIC is active what is still in ISA mode (Edge sensitive, High active)? Should the RTC be handled differently than the Keyboard?

Anyone else out there getting interrupts from the RTC via the I/O APIC in VMware?

UPDATE: If I set the RTC I/O APIC entry to Level sensitive, Low active then it will fire once! However after that the system hangs.

-Ian

[Brendan]

Hi,

Do I still need to handle the cascade IRQ (2) when using the I/O APIC? I would assume not.

You shouldn't need to care about the PIC's cascade (and shouldn't need an IRQ handler for it).

You do need to handle spurious IRQs from the PIC properly though (both IRQ 7 and IRQ 15) when using the PICs and when using IO APICs (because even though you're using IO APICs, you can't mask the PIC's spurious IRQs). This includes detecting the difference between spurious IRQs and real IRQs when the PIC is being used.

Is it possible that the RTC is being mapped to a different interrupt number (like the legacy timer is moved from 0 to 2) and I am missing it?

It can be mapped to a different IO APIC input number, but you shouldn't be missing it if you're using the ACPI MADT or MP Spec tables properly.

Any ideas would be greatly appreciated.

Is a "permanently in progress" higher priority interrupt blocking delivery of lower priority interrupts?

Is there a bug in the ACPI MADT or MP Spec parsing code?

Is there a bug in the IO APIC setup code?

There are bugs in the old version of the RTC initialisation code (v0.5.2 from your web site), but these bugs shouldn't be causing problems with the periodic IRQ (where "shouldn't" doesn't imply that putting the RTC into a dodgy state won't confuse an emulator completely and cause other problems). Not sure if the new version of the code is different.

Note: Don't change the RTC to "24 hour, binary mode" because some (most?) systems don't reset the RTC's mode properly during boot and the BIOS/firmware gets all confused when it thinks the RTC is in BCD mode and it's not (turning the computer off won't help either, as the RTC remembers). If you must do it, do it right - read the time and date and decode it according to the firmware's mode, then change the RTC's operating mode, then reprogram the time and date to match the new operating mode (so you don't end up with "33 o'clock on the 50th day of the 17th month") and then do the reverse to restore whatever mode the firmware was using before any reboot or shutdown (including before any triple fault, and before the user presses any reset or power switch).


Cheers,

Brendan

[rdos]

Obviously, my APIC code malfunctions on a single computer (ASUS motherboard with 6-core AMD phenom), while it works on at least 4 other multicore computers. Currently, I have no idea why, but it seems like there is no interrupt from any APIC source at all, which includes the timer. When I use the PIC on this computer, everything works as expected.

Edit: I have now confirmed that no IRQs are firing, and that this is why the system stops. I added a print "A" before the hlt instruction in the null-thread, and a print "B" after it, and only the A is displayed on screen, and the keyboard doesn't respond. It feels like this might also be why the Compaq CQ57 is missing interrupts in both the SATA and network device. The strange thing is that 3 other machines have no interrupt issues.

It turned out that the initialization code set TPR to FF, and then the APIC timer never fires. However, after initializing TPR to 0, the other interrupts seem to work, but the APIC timer still doesn't fire.

[IanSeyler]

Hi Brendan,

Is a "permanently in progress" higher priority interrupt blocking delivery of lower priority interrupts?

Is there a bug in the ACPI MADT or MP Spec parsing code?

Is there a bug in the IO APIC setup code?

There are bugs in the old version of the RTC initialisation code (v0.5.2 from your web site), but these bugs shouldn't be causing problems with the periodic IRQ (where "shouldn't" doesn't imply that putting the RTC into a dodgy state won't confuse an emulator completely and cause other problems). Not sure if the new version of the code is different.

How would I find out about a high priority interrupt? I thought the priority list started as 0,1,2,8,... If that is still the case then I am getting interrupts via 1 (keyboard) and 2 (timer). Everything else is masked (on the PIC as well).

The parsing code seems fine but I am unsure about the I/O APIC setup. For instance I'm not quite sure what to do with the spurious vector (I just set it to 0xF8). Also I'm not sure what to do with LINT0 and LINT1 (I have both set to a vector of 0). The current code is here: http://code.google.com/p/pure64/source/ ... it_smp.asm

As for the RTC initialization code you are correct. The older code I used caused issues with the BIOS losing its settings after a reboot.

Thanks,
-Ian

[Brendan]

Hi,

Is a "permanently in progress" higher priority interrupt blocking delivery of lower priority interrupts?

Is there a bug in the ACPI MADT or MP Spec parsing code?

Is there a bug in the IO APIC setup code?

There are bugs in the old version of the RTC initialisation code (v0.5.2 from your web site), but these bugs shouldn't be causing problems with the periodic IRQ (where "shouldn't" doesn't imply that putting the RTC into a dodgy state won't confuse an emulator completely and cause other problems). Not sure if the new version of the code is different.

How would I find out about a high priority interrupt? I thought the priority list started as 0,1,2,8,... If that is still the case then I am getting interrupts via 1 (keyboard) and 2 (timer). Everything else is masked (on the PIC as well).

The "0,1,8,9,10,11,12,13,14,15,3,4,5,6,7" order only applies to the PIC. IRQ priorities for IO APICs and local APICs depends on the interrupt vector for each interrupt, not which IO APIC input (or PIC input) it uses.

You should be able to look at the local APIC's "In Service Register" to determine if any IRQs from the IO APIC or local APIC are in service; but you may need to do this on each CPU's local APIC.

The parsing code seems fine but I am unsure about the I/O APIC setup. For instance I'm not quite sure what to do with the spurious vector (I just set it to 0xF8).

APIC's spurious vector can be anything (I'd choose the lowest priority interrupt vector though; and I'd normally use a vector number that has the lowest 4 bits set in case you ever feel like supporting older 32-bit 80x86 CPUs).

Also I'm not sure what to do with LINT0 and LINT1 (I have both set to a vector of 0). The current code is here: http://code.google.com/p/pure64/source/ ... it_smp.asm

LINT0 and LINT1 are typically "NMI" and "extInt". Unless the ACPI or MP spec tables tell you exactly what to do with them don't touch them.

For the "init_smp.asm" code you linked to:

• When searching for the Root System Description Pointer Structure, if you find the "RSD PTR " you may have found the RSDP but you might not have - you have to check the checksum before you know if it actually is the RSDP or if it was just a coincidence.
• For the Spurious Interrupt Register, Focus Processor Checking isn't supported on modern CPUs and you shouldn't set the bit or attempt to enable Focus Processor Checking on CPUs that don't support it.
• For the Spurious Interrupt Register, EOI-Broadcast Suppression isn't supported on older CPUs (I think it was introduced with x2APIC - not sure). You shouldn't set the bit or attempt to enable EOI-Broadcast Suppression on CPUs that don't support it; and you shouldn't set the bit or attempt to enable EOI-Broadcast Suppression unless the OS is also using the "directed EOI" feature (where the OS sends the EOI to the local APIC *and* to the correct IO APIC).
• I don't know why you're setting the interrupt vector to zero in the local APIC's "LVT Error Register". Interrupt vector 0 is reserved for the "Divide Error" exception, and all interrupt vectors used by the local APIC or IO APIC should be 32 or higher.
• The IO APIC initialisation is dodgy:
  • you assume there's only one IO APIC. I've got computers with 2 IO APICs and I've seen computers with four of them. "IO APIC input 33" might be the second input on the third IO APIC.
  • you assume that keyboard is connected to IO APIC Input 1 and assume that RTC is connected to IO APIC input 8. You can't make assumptions like this. You need to parse the ACPI MADT/APIC table (or the MP Spec table) properly to determine which ISA IRQs are mapped to which IO APIC inputs. For all you know RTC might be mapped to "IO APIC input 33, the second input of the third IO APIC".
  • in the same way, you don't set the trigger mode or polarity according to what the ACPI or MP Spec tables told you to use.
  • you're using "physical delivery" and send the IRQs to "local APIC ID zero", even when there are no CPUs with "local APIC ID zero". Nobody said that the BSP is "local APIC ID zero" or that local APIC IDs are contiguous.

I didn't look at any of the AP CPU startup, or speed determination, or anything in "init_smp_acpi.asm".


Cheers,

Brendan