I’ve really been looking forward to playing a new game (‘Battlefleet Gothic Armada’ set in the Warhammer 40k universe) so when the beta was released to pre-order customers I was very much looking forward to a new game set in one of my favourite intellectual properties.
Unfortunately I’ve been having no end of crashes with little to go on as to the cause – obviously beta software has issues and I’m more than happy to help with feedback but at the rate I was experiencing crashes it didn’t look likely I’d have any chance. Thankfully the last crash left me with a dump file which I grabbed and set about working on in Microsoft’s WinDbg software. This is a bit of a deviation from my normal posting but it’s a great way to show how tools used diagnosing crash dumps at work can be leveraged at home.
To make sure we can properly diagnose the problem we need to provide the symbol path – in this case I am using the Microsoft public symbol server and instructing WinDbg to cache a copy at the location C:\Symbols. It should be noted that you may have problems where symbols are not made available – this is just one of those things you have to live with unfortunately in some cases.
It may take some time for the symbols to download and cache so patience is a virtue. The next step is to open the crash dump file to begin analysis.
************* Symbol Path validation summary ************** Response Time (ms) Location Deferred .sympath srv*c:\Symbols*https://msdl.microsoft.com/download/symbols Deferred srv*c:\symbols*https://msdl.microsoft.com/download/symbols Microsoft (R) Windows Debugger Version 10.0.10586.567 AMD64 Copyright (c) Microsoft Corporation. All rights reserved. Loading Dump File [C:\Users\A\Desktop\UE4Minidump.dmp] User Mini Dump File: Only registers, stack and portions of memory are available ************* Symbol Path validation summary ************** Response Time (ms) Location Deferred .sympath srv*c:\Symbols*https://msdl.microsoft.com/download/symbols Deferred srv*c:\symbols*https://msdl.microsoft.com/download/symbols Symbol search path is: .sympath srv*c:\Symbols*https://msdl.microsoft.com/download/symbols;srv*c:\symbols*https://msdl.microsoft.com/download/symbols Executable search path is: Windows 8.1 Version 9600 MP (12 procs) Free x64 Product: WinNt, suite: SingleUserTS Built by: 6.3.9600.18217 (winblue_ltsb.160124-0053) Machine Name: Debug session time: Sun Mar 27 19:59:52.000 2016 (UTC + 1:00) System Uptime: not available Process Uptime: 0 days 0:00:22.000 ................................................................ ................................................................ ...... This dump file has an exception of interest stored in it. The stored exception information can be accessed via .ecxr. (3ac0.3a10): Access violation - code c0000005 (first/second chance not available) ntdll!NtGetContextThread+0xa: 00007ffe`61ba14ca c3 ret
You may notice WinDbg indicates exception information is available and can be access by using the .ecxr command – if you are wondering this command will display the context record associated with the current exception. Enter the command, hit enter and see what comes back.
As you can see WinDbg couldn’t find a symbol required – this is to be expected when dealing with software from other vendors.
0:079> .ecxr
*** ERROR: Symbol file could not be found. Defaulted to export symbols for nvwgf2umx.dll -
rax=0000000010000000 rbx=0000002338439f50 rcx=0000002331252e70
rdx=0000000080000000 rsi=0000000000000000 rdi=0000002331252e70
rip=00007ffe5bf26d16 rsp=00000023504ee2e0 rbp=0000000000000001
r8=000000230bc40330 r9=0000000000000027 r10=00000023401a4010
r11=0000000000000000 r12=0000000000000000 r13=0000000000000001
r14=0000000000000000 r15=0000002b38364d90
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
nvwgf2umx!NVAPI_Thunk+0x29b606:
00007ffe`5bf26d16 41816708ffffff0f and dword ptr [r15+8],0FFFFFFFh ds:0000002b`38364d98=????????
I decided to get some extended information by running the command !analyze –v
This will give us a lot of output and you will see messages relating to symbol files not being -
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
*** ERROR: Symbol file could not be found. Defaulted to export symbols for GameOverlayRenderer64.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for BattleFleetGothic-Win64-Shipping.exe -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for tmmon64.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for TmUmEvt64.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for tier0_s64.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for steamclient64.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for Leap.dll -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for WMVDECOD.DLL -
*** ERROR: Symbol file could not be found. Defaulted to export symbols for nvspcap64.dll -
*** WARNING: Unable to verify checksum for RakNet120_Release_x64.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for RakNet120_Release_x64.dll -
*** WARNING: Unable to verify checksum for Stormancer120_Release_x64.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for Stormancer120_Release_x64.dll -
DUMP_CLASS: 2
DUMP_QUALIFIER: 400
CONTEXT: (.ecxr)
rax=0000000010000000 rbx=0000002338439f50 rcx=0000002331252e70
rdx=0000000080000000 rsi=0000000000000000 rdi=0000002331252e70
rip=00007ffe5bf26d16 rsp=00000023504ee2e0 rbp=0000000000000001
r8=000000230bc40330 r9=0000000000000027 r10=00000023401a4010
r11=0000000000000000 r12=0000000000000000 r13=0000000000000001
r14=0000000000000000 r15=0000002b38364d90
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
nvwgf2umx!NVAPI_Thunk+0x29b606:
00007ffe`5bf26d16 41816708ffffff0f and dword ptr [r15+8],0FFFFFFFh ds:0000002b`38364d98=????????
Resetting default scope
FAULTING_IP:
nvwgf2umx!NVAPI_Thunk+29b606
00007ffe`5bf26d16 41816708ffffff0f and dword ptr [r15+8],0FFFFFFFh
EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 00007ffe5bf26d16 (nvwgf2umx!NVAPI_Thunk+0x000000000029b606)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000001
Parameter[1]: 0000002b38364d98
Attempt to write to address 0000002b38364d98
DEFAULT_BUCKET_ID: INVALID_POINTER_WRITE
PROCESS_NAME: BattleFleetGothic-Win64-Shipping.exe
ERROR_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%08lx referenced memory at 0x%08lx. The memory could not be %s.
EXCEPTION_CODE_STR: c0000005
EXCEPTION_PARAMETER1: 0000000000000001
EXCEPTION_PARAMETER2: 0000002b38364d98
WRITE_ADDRESS: 0000002b38364d98
FOLLOWUP_IP:
nvwgf2umx!NVAPI_Thunk+29b606
00007ffe`5bf26d16 41816708ffffff0f and dword ptr [r15+8],0FFFFFFFh
BUGCHECK_STR: INVALID_POINTER_WRITE
WATSON_BKT_PROCSTAMP: 56f50c06
WATSON_BKT_PROCVER: 4.9.2.0
PROCESS_VER_PRODUCT: Unreal Engine
WATSON_BKT_MODULE: nvwgf2umx.dll
WATSON_BKT_MODSTAMP: 56de6386
WATSON_BKT_MODOFFSET: 956d16
WATSON_BKT_MODVER: 10.18.13.6451
MODULE_VER_PRODUCT: NVIDIA D3D10 drivers
BUILD_VERSION_STRING: 6.3.9600.17415 (winblue_r4.141028-1500)
MODLIST_WITH_TSCHKSUM_HASH: 1675b72eddb10cabfc95badd466638890e1076e2
MODLIST_SHA1_HASH: 47fe1a1c8831d62294704a5d22c69e6eabde4359
DUMP_FLAGS: 0
DUMP_TYPE: 2
APP: battlefleetgothic-win64-shipping.exe
ANALYSIS_SESSION_HOST: AG
ANALYSIS_SESSION_TIME: 03-28-2016 22:54:26.0875
ANALYSIS_VERSION: 10.0.10586.567 amd64fre
THREAD_ATTRIBUTES:
PROBLEM_CLASSES:
INVALID_POINTER_WRITE
Tid [0x3a10]
Frame [0x00]: nvwgf2umx!NVAPI_Thunk
LAST_CONTROL_TRANSFER: from 00007ffe5bce29fe to 00007ffe5bf26d16
STACK_TEXT:
00000023`504ee2e0 00007ffe`5bce29fe : 00000000`00000001 00000023`504ee8a0 00000000`00000000 00000000`00000000 : nvwgf2umx!NVAPI_Thunk+0x29b606
00000023`504ee350 00007ffe`5bce2775 : 00000023`31252e70 00000000`80000000 00000000`00000001 00000023`504ee509 : nvwgf2umx!NVAPI_Thunk+0x572ee
00000023`504ee380 00007ffe`5bce28bf : 00000000`00000000 00000023`325c2800 00000023`383556e0 00000023`504ee8a0 : nvwgf2umx!NVAPI_Thunk+0x57065
00000023`504ee3b0 00007ffe`5be280e2 : 00000023`38439e40 00000000`00000000 00000000`00000000 00000000`00000000 : nvwgf2umx!NVAPI_Thunk+0x571af
00000023`504ee3e0 00007ffe`5be254c7 : 00000049`08363f4e 00000023`325c2800 00000000`00000000 00000000`00000000 : nvwgf2umx!NVAPI_Thunk+0x19c9d2
00000023`504ee410 00007ffe`5b5f02e2 : 00000023`504ee860 00000023`504ee650 00000023`504ee8a0 00000000`00000000 : nvwgf2umx!NVAPI_Thunk+0x199db7
00000023`504ee550 00007ffe`5b706ff8 : 00000002`00001000 00000000`00000000 00000000`00000000 00000023`504ee808 : nvwgf2umx!OpenAdapter12+0x157e2
00000023`504ee720 00007ffe`5b716dbc : 00000023`504ee990 00000000`00000000 00000023`00000000 00000023`504ee808 : nvwgf2umx!OpenAdapter12+0x12c4f8
00000023`504ee7b0 00007ffe`5b664158 : 00000023`3261bda0 00000024`88238b00 00000023`504eeaa0 00000780`ffffffff : nvwgf2umx!OpenAdapter12+0x13c2bc
00000023`504ee960 00007ffe`5cbb3c7b : 00000023`00000000 00000000`00000001 00000000`00000000 00000000`00000000 : nvwgf2umx!OpenAdapter12+0x89658
00000023`504eea20 00007ffe`5c8a2a24 : 00007ffe`5cbb2a00 00007ffe`5cbb3670 00000023`504eee50 00000023`383403d0 : d3d11!NDXGI::CDevice::Present+0x5a8
00000023`504eec10 00007ffe`5c8acb8f : 00000023`504ef7e0 00000023`38337990 00000000`00000000 00000000`00000000 : dxgi!CDXGISwapChain::PresentImplCore+0xb59
00000023`504ef570 00007ffe`5c8ace69 : 00007f00`00000000 00000000`00000000 00000023`504ef6a9 00000000`00000000 : dxgi!CDXGISwapChain::PresentImpl+0xff
00000023`504ef630 00007ffe`41a261e3 : 00000023`3ff11760 400ccccd`3f2fabb8 00000023`504ef848 00000023`504ef840 : dxgi!CDXGISwapChain::Present+0x89
00000023`504ef710 00007ff6`8a728262 : 00000023`46938101 00000023`469381c0 00000000`00001b00 00000000`00000000 : GameOverlayRenderer64!OverlayHookD3D3+0x16ae3
00000023`504ef740 00007ff6`8a728207 : 00000023`469381c0 00000000`00000000 00000000`00000000 00000023`469381c0 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x8ab632
00000023`504ef780 00007ff6`8a72ec9b : 00000024`880a72a0 00000000`00000001 00000000`00000010 00000023`33b00008 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x8ab5d7
00000023`504ef7d0 00007ff6`8a7c3cf6 : 00000000`00000000 00000023`00000000 00007ff6`8cb112d8 00007ff6`8cb112d8 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x8b206b
00000023`504ef840 00007ff6`8aaf75f3 : 00000023`481f4680 00000023`34483200 00000023`504ef970 00007ff6`8cb112d8 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x9470c6
00000023`504ef870 00007ff6`8aaf8312 : 00000023`48358000 00000024`21c7e800 00000024`21c7e900 000004b0`00000780 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0xc7a9c3
00000023`504efa90 00007ff6`8a2799b7 : 00000000`0000005c 00000023`32bbf9c8 00000023`32bbf9c8 00000023`32bbf9c8 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0xc7b6e2
00000023`504efad0 00007ff6`8a279a2d : 00000000`00000000 00007ffe`5ed3155c 00000000`00000001 00000023`32bbf9c8 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x3fcd87
00000023`504efb40 00007ff6`8a7d248a : 00000023`48960b40 00000023`46d563b0 00000000`00000000 00000000`00000000 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x3fcdfd
00000023`504efb80 00007ff6`8a7d24ef : 00000023`46d563b0 00007ffe`5ed3155c 00000000`00000000 00000000`00000000 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x95585a
00000023`504efbb0 00007ff6`8a32aac7 : 00000023`46d563b0 00000023`46d563b0 00000000`00000000 00000000`00000000 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x9558bf
00000023`504efbf0 00007ff6`8a327a04 : 00000023`46d563b0 00000000`00000000 00000000`00000000 00000000`00000000 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x4ade97
00000023`504efc20 00007ffe`609613d2 : 00000023`46d563b0 00007ff6`8a323990 00000000`00000000 00000000`00000000 : BattleFleetGothic_Win64_Shipping!opus_repacketizer_out_range+0x4aadd4
00000023`504efc60 00007ffe`61b254e4 : 00007ffe`609613b0 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x22
00000023`504efc90 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x34
THREAD_SHA1_HASH_MOD_FUNC: 79851e6c71c9d39e9c069770304adcc9e44e987c
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 305cd78fdde18f2ed34cf607b65623a24679b60c
THREAD_SHA1_HASH_MOD: 73bacbffa7804d3f769606f841c547c6734420b0
FAULT_INSTR_CODE: 8678141
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: nvwgf2umx!NVAPI_Thunk+29b606
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: nvwgf2umx
IMAGE_NAME: nvwgf2umx.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 56de6386
STACK_COMMAND: .ecxr ; kb
BUCKET_ID: INVALID_POINTER_WRITE_nvwgf2umx!NVAPI_Thunk+29b606
PRIMARY_PROBLEM_CLASS: INVALID_POINTER_WRITE_nvwgf2umx!NVAPI_Thunk+29b606
BUCKET_ID_OFFSET: 29b606
BUCKET_ID_MODULE_STR: nvwgf2umx
BUCKET_ID_MODTIMEDATESTAMP: 56de6386
BUCKET_ID_MODCHECKSUM: 12282fb
BUCKET_ID_MODVER_STR: 10.18.13.6451
BUCKET_ID_PREFIX_STR: INVALID_POINTER_WRITE_
FAILURE_PROBLEM_CLASS: INVALID_POINTER_WRITE
FAILURE_EXCEPTION_CODE: c0000005
FAILURE_IMAGE_NAME: nvwgf2umx.dll
FAILURE_FUNCTION_NAME: NVAPI_Thunk
BUCKET_ID_FUNCTION_STR: NVAPI_Thunk
FAILURE_SYMBOL_NAME: nvwgf2umx.dll!NVAPI_Thunk
FAILURE_BUCKET_ID: INVALID_POINTER_WRITE_c0000005_nvwgf2umx.dll!NVAPI_Thunk
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/BattleFleetGothic-Win64-Shipping.exe/4.9.2.0/56f50c06/nvwgf2umx.dll/10.18.13.6451/56de6386/c0000005/00956d16.htm?Retriage=1
TARGET_TIME: 2016-03-27T18:59:52.000Z
OSBUILD: 9600
OSSERVICEPACK: 17415
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK: 256
PRODUCT_TYPE: 1
OSPLATFORM_TYPE: x64
OSNAME: Windows 8.1
OSEDITION: Windows 8.1 WinNt SingleUserTS
OS_LOCALE:
USER_LCID: 0
OSBUILD_TIMESTAMP: 2014-10-29 02:45:30
BUILDDATESTAMP_STR: 141028-1500
BUILDLAB_STR: winblue_r4
BUILDOSVER_STR: 6.3.9600.17415
ANALYSIS_SESSION_ELAPSED_TIME: 8efb
ANALYSIS_SOURCE: UM
FAILURE_ID_HASH_STRING: um:invalid_pointer_write_c0000005_nvwgf2umx.dll!nvapi_thunk
FAILURE_ID_HASH: {394d2a7d-b34b-6afc-ef2e-834df1b40588}
Followup: MachineOwner
WinDbg provides a link for the ‘MODULE NAME’ section – if you click the link it will cause information to be displayed –
0:079> lmvm nvwgf2umx Browse full module list start end module name 00007ffe`5b5d0000 00007ffe`5c7e2000 nvwgf2umx (export symbols) nvwgf2umx.dll Loaded symbol image file: nvwgf2umx.dll Mapped memory image file: C:\Windows\System32\nvwgf2umx.dll Image path: C:\Windows\System32\nvwgf2umx.dll Image name: nvwgf2umx.dll Browse all global symbols functions data Timestamp: Tue Mar 08 05:30:46 2016 (56DE6386) CheckSum: 012282FB ImageSize: 01212000 File version: 10.18.13.6451 Product version: 10.18.13.6451 File flags: 8 (Mask 3F) Private File OS: 40004 NT Win32 File type: 3.4 Driver File date: 00000000.00000000 Translations: 0409.04e4 CompanyName: NVIDIA Corporation ProductName: NVIDIA D3D10 drivers InternalName: nvwgf2um OriginalFilename: nvwgf2um.dll ProductVersion: 10.18.13.6451 FileVersion: 10.18.13.6451 FileDescription: NVIDIA D3D10 Driver, Version 364.51 LegalCopyright: (C) 2016 NVIDIA Corporation. All rights reserved.
Now I’m not going to expect you to have read all the output so the short of it is it looks like the issue exists with the NVIDIA 3D display driver. By chance an update was released today so I decided I would install the latest drivers but with the 3D drivers removed from my system as I don’t have any 3D hardware (monitor/headset etc).
I went digging around on the forums setup for the game and found a mention that these drivers should be removed so it looks like I was on the right track. Of course I could have spent time searching the forums for any articles on this but it was more fun to do it myself and it is always nice to practice those debug skills. So far the game hasn’t crashed in the same way it was previously so fingers crossed the issue was resolved by un-installing those 3D drivers.
Cool run through man, found this from googling opus_repacketizer_out_range in a particular windbg error. Not sure what that specifically means but will fuss with my drivers to fix.