Compiling Gideros Luau Source

From GiderosMobile
Revision as of 17:06, 30 January 2022 by MoKaLux (talk | contribs)

The following instructions are for developers who want to contribute to or otherwise experiment with the Gideros C/C++ engine and are not necessary for either building or deploying apps made with Gideros.

The Gideros project consists of the Gideros Studio IDE, the Gideros SDK libraries, Gideros Texture Packer, Gideros Font Creator, "players" or export templates for all the platforms Gideros supports, all targets of all plugins, a daemon and a bridge. The build system is flexible, so you only need the tools for the targets you are interested in deploying to with Qt (and MSYS2 on Windows) being a realistic minimum.

Requirements for Windows 10 64bit

Install QT

You can download the Qt for open source users at:

https://www.qt.io/download

Recommended version 6.2.3 with MinGW 9.0

Double click on the downloaded file and follow instructions. When installing:

  • change the install directory to /Qt rather than /Qt6 (optional)
  • expand the tree view and ensure you have ticked the MinGW child option of the Qt version checkbox, as well as the MinGW option in the Tools section
  • Qt 5 compatibility module
  • and following additional Qt libraries:
    • Qt Multimedia
    • Qt Websockets

If you already have Qt6 installed without those modules, you can add them via Qt MaintenanceTool.exe located in your Qt folder like so:

(to add capture)

Here is a screenshot of the Qt modules:

(to add capture)

When you are done click finish.


Install MSYS2 Console

https://www.msys2.org/

Once installed, open MSYS2 shell (msys2_shell.cmd):

(to add capture)

Then run the following commands:

note: copy/paste won’t work, you have to type the commands 

pacman –Sy pacman
pacman –Syu
pacman –Su
pacman –S tar
pacman –S zip
pacman –S unzip
pacman –S git
pacman –S base-devel
pacman -S python

#For compiling win32 under a MinGW64
pacman -S mingw-w64-i686-toolchain
pacman -S mingw-w64-i686-glew

note: for pacman -S mingw-w64-i686-toolchain, if asked choose group 17?


Install Android Studio and NDK

https://developer.android.com/studio/index.html

https://developer.android.com/ndk/downloads/older_releases.html (old for gideros ndk 17c)

https://github.com/android/ndk/wiki (for newer releases)

Recommended NDK is 17c (the x86_64 version)

Once downloaded, copy the contents of the NDK to your home or C: folder, rename the folder that you copied to 'android-ndk', for example you will end up with 'c:\android-ndk' and below that folders that say build, meta, platforms, etc.


Install JAVA

You need the Java JDK from Oracle:

https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

Or an open JDK (recommanded):

https://adoptopenjdk.net/


Install Visual Studio

Visual Studio Community Edition 2022:

https://www.visualstudio.com/downloads/

Ensure that you have MSBuild v12 or v14 installed (building with the makefile will fail with an incorrect version).

– to come -- For UWP ads: https://social.msdn.microsoft.com/Forums/windowsapps/en-US/db8d44cb-1381-47f7-94d3-c6ded3fea36f/microsoft-ad-monetization-platform-shut-down-as-of-june-1st?forum=aiamgr – to come --


Download Scintilla

from Gideros github repository (code/download zip):

https://github.com/gideros/scintilla

(original zip file from https://www.scintilla.org/ScintillaDownload.html)

Then you will need to compile the .dll for scintilla:

  • navigate to ...\scintilla-master\qt\ScintillaEdit\
  • open ScintillaEdit.pro (this will launch Qt6 or you can open it in Qt6)
  • in Qt6 click on Projects and validate the Build Settings
  • in the menu choose Build and Build Project “ScintillaEdit”
  • note that you can build for Debug and for Release in the Qt6 Projects tab
  • after compilation you should have your ScintillaEdit5.dll in ...\scintilla-master\qt\build-ScintillaEdit-Desktop_Qt_6_2_3_MinGW_64_bit-Debug\debug and ...\scintilla-master\qt\build-ScintillaEdit-Desktop_Qt_6_2_3_MinGW_64_bit-Release\release


Download Lexilla

from Gideros github repository (code/download zip):

https://github.com/gideros/lexilla

(original zip file from https://www.scintilla.org/LexillaDownload.html)

Then you also need to compile the .dll for lexilla:

  • copy scintilla header files from ‘’’...\scintilla-master\include’’’ to ‘’’...\lexilla-master\include’’’
  • open C:\dev\lexilla-master\src\Lexilla.pro (this will launch Qt6 or you can open it in Qt6)
  • in Qt6 click on Projects and validate the Build Settings
  • in the menu choose Build and Build Project “Lexilla”
  • note that you can build for Debug and for Release in the Qt6 Projects tab
  • after compilation you should have your lexilla5.dll in ...\C:\dev\lexilla-master\build-Lexilla-Desktop_Qt_6_2_3_MinGW_64_bit-Debug\debug and ...\C:\dev\lexilla-master\build-Lexilla-Desktop_Qt_6_2_3_MinGW_64_bit-Release\release


Install Python

https://www.python.org/downloads/release/python-2712/

You need Python 2.7.12 or later to install emscripten properly.

Once installed, add it to your command line path, eg: c:\python27. On windows you can do this by right-clicking on 'myComputer' or 'This PC', go to 'Properties', click 'Advanced System Settings', Click 'Environmental Variables', double-click 'Path', add 'c:\python27'


Install emscripten

http://kripken.github.io/emscripten-site/

You need this if you want to build for HTML5 + Facebook games.

Using a shell or MSYS2 (previously installed) on windows, go to your home or C: folder and type:

Once that has completed open a command prompt in that folder on Windows.

Using emsdk, install and activate all the latest packages:

note: (on non-PC put ./ before emsdk) 

emsdk install latest
emsdk activate latest
emsdk_env.bat

To use incoming branch of emscripten, use these commands instead:

note: (on non-PC put ./ before emsdk) 

emsdk install sdk-incoming-64bit
emsdk activate sdk-incoming-64bit

To further use Gideros own git fork of emscripten: 

cd emscripten/incoming
#Add a git remote link to your own repository
git remote add gideros https://github.com/gideros/emscripten.git
#Obtain the changes in your link
git fetch gideros
#Switch the emscripten-incoming tool to use your fork
git checkout -b gideros_incoming --track gideros/incoming
#Get back to emscripten directory and rebuild
cd ../..
emsdk install sdk-incoming-64bit

These can also be installed within emsdk: git, crunch, and spidermonkey:

note: (on non-PC put ./ before emsdk) 

emsdk install git-1.9.4
emsdk install crunch-1.03
emsdk install spidermonkey-37.0.1-64bit
emsdk activate git-1.9.4
emsdk activate crunch-1.03
emsdk activate spidermonkey-37.0.1-64bit

(PC) 

emsdk_env.bat

(PC) Make sure the environment variable EM_CONFIG is set to the location of your emscripten config file.

The emscripten build also needs the CMake command. You can download it and extract it to your home or C: folder. Don't forget to add it's path to the end of the EMSDKPATH in Makefile.def.

https://cmake.org/download/


Install Steam SDK (optional)

https://partner.steamgames.com/?goto=%2Fdownloads%2Fsteamworks_sdk.zip

Extract the archive/zip file to your home or C: folder, for example if the file is called 'steamworks_sdk_142.zip' then you will have a 'c:\steamworks_sdk_142' folder with a solder called 'sdk' within it. The sdk folder contains other folders such as glmgr, public, tools, etc.

This is necessary for the Steam plugin.


Install NSIS

https://nsis.sourceforge.io/Download


Install Sign Tool

'’’???’’’


Install Gideros from Source

Fetch gideros source code

(PC) Open MSYS2 32 bit shell

Navigate to the folder where you want to work. 

git clone https://github.com/gideros/gideros

Configure scripts

Make a copy of the file gideros/scripts/Makefile.def.example, rename it to Makefile.def and update the file copy according to your installation. Ensure that the QT\_ARCH and QT_TOOLSARCH paths are formatted correctly according to your Qt install directory structure (see below that they are named differently by Qt). Example:

On PC: For Qt: 

QTBASEDIR=/c/Qt
QT_ARCH=mingw53_32
QT_TOOLSARCH=mingw530_32
QTVER=5.9.1
QT5ICUVER=59

# for steam:
STEAMSDK=/c/Applications/steamworks_sdk_142
export STEAMSDK

# for Android:
# on Windows, also add this to PATH
NDK_HOME=/c/android/android-ndk-r17b
NDKBUILD=cmd //c /c/android/android-ndk-r17b/ndk-build.cmd
# We use windows syntax for this one, as it will be used by gradle.bat script
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_144
# Windows syntax with quadruple backslashs!
ANDROID_HOME=C:\\\\Users\\\\Gideros\\\\AppData\\\\Local\\\\Android\\\\sdk

# for WinRT:
FXC="/c/Program Files (x86)/Windows Kits/8.1/bin/x86/fxc.exe"
MSBUILD="/c/Program Files (x86)/MSBuild/14.0/Bin/MSbuild.exe"

# for html5:
EMSDK=/c/emsdkwin
EMSDKPATH=$(EMSDK):$(EMSDK)/clang/e1.35.0_64bit:$(EMSDK)/node/4.1.1_64bit/bin:$(EMSDK)/python/2.7.5.3_64bit:$(EMSDK)/emscripten/1.35.0
EMSDK_PREFIX=PATH=$(EMSDKPATH):$$$$PATH cmd //c
CRUNCHME=crunchme-win32.exe


Complete working example Makefile.def for Windows

Here is a working example, you will have to change the filename details to match your setup: example Makefile.def file 

QTBASEDIR=/c/Qt

#export PATH=$PATH:~/Android_SDK/NDK/

# on Windows, also add this to PATH
NDK_HOME=/c/android-ndk

NDKBUILD=cmd //c /c/android-ndk/ndk-build.cmd

# remember to escape disallowed characters, or wrap in quotes
JAVA_HOME=C:\Program Files (x86)\Java\jdk1.8.0_181
ANT_HOME=/c/apache-ant-1.10.5
ANDROID_HOME=C:\\\\Users\\\\antho\\\\AppData\\\\Local\\\\Android\\\\Sdk

FXC="/c/Program Files (x86)/Windows Kits/8.1/bin/x86/fxc.exe"
FXCx64="/c/Program Files (x86)/Windows Kits/8.1/bin/x64/fxc.exe"
MSBUILD="/c/Program Files (x86)/MSBuild/14.0/Bin/MSBuild.exe"

EMSDK=/c/emsdk
EMSDKPATH=$(EMSDK):$(EMSDK)/clang/e1.38.12_64bit:$(EMSDK)/node/8.9.1_64bit/bin:$(EMSDK)/python/2.7.13.1_64bit/python-2.7.13.amd64:$(EMSDK)/emscripten/1.38.12:/c/cmake-3.12.3-win32-x86/bin
EMSDK_PREFIX=PATH=$(EMSDKPATH):$$$$PATH

#EMSDK_PREFIX+=BINARYEN_ROOT=$(EMSDK)/clang/e1.38.12_64bit/binaryen
#EMSDK_PREFIX+=EMSDK=$(EMSDK)
#EMSDK_PREFIX+=EMSCRIPTEN=$(EMSDK)/emscripten/1.38.12
#EMSDK_PREFIX+=EMSCRIPTEN_ROOT=$(EMSDK)/emscripten/incoming
#EMSDK_PREFIX+=JAVA=$(EMSDK)/emscripten/java/8.152_64bit/bin/java.exe
EMSDK_PREFIX+=cmd //c

CRUNCHME=crunchme-qt-win.exe

STEAMSDK=/c/steamworks_sdk_142
export STEAMSDK

NSIS=cmd //c "/c/Program Files (x86)/NSIS/makensis.exe"
WIN_SIGN="/c/Program Files (x86)/Windows Kits/10/bin/x86/signtool.exe"
WIN_KEYSTORE=/c/comodo-code-signing.p12
WIN_KEYPASS=xxxx
WIN_KEYTSS="http://timestamp.comodoca.com"

MAC_HOST=user@host
MAC_PATH=gideros/gideros
OSX_SIGNING_IDENTITY="ID"
OSX_SIGNING_PASSWORD="keychain password"

QT_ARCH=mingw53_32
QT_TOOLSARCH=mingw530_32
QTVER=5.10.1
QT5ICUVER=59  #deprecated stuff


Fetch additional repositories

Go into just cloned gideros source folder and run make prep.repo 

cd gideros
make -f scripts/Makefile.gid prep.repo


Building

If you have all dependencies in place for all targets, you can build everything at once via MSYS2 or Terminal with: 

make -f scripts/Makefile.gid


Compilation on PC

Open MSYS2 shell (msys2_shell.cmd). Standard compilation is 32 bit so use a 32 bit shell. Make sure you have installed the necessary build tools in MSYS2 using pacman, as detailed in Install MSYS2 Console

Navigate to gideros folder

Build QT tools:

make –f scripts/Makefile.gid qtapp.install

=> files GiderosStudio.exe, GiderosPlayer.exe, GiderosTexturePacker.exe (…) in Build.Win folder

Android:

make –f scripts/Makefile.gid android.install

=> file GiderosAndroidPlayer.apk in folder Build.Win/Players

winRT:

make –f scripts/Makefile.gid winrt.install

=> file _bundle.appxupload in folder Build.Win/Players/WinRT

Note: if necessary, open the solution, gideros/winRT_example/giderosgame.sln in Visual Studio. Check that the SQLite version for Windows and Windows Phone is up to date. Save and close the solution.

win32:

make –f scripts/Makefile.gid win32.install

=> files in folder Build.Win/Templates/win32

Html5:

make –f scripts/Makefile.gid html5.install

=> files in folders Build.Win//Players/Html5 and Build.Win/Templates/Html5

Plugins:

make –f scripts/Makefile.gid bundle.win

=> folder bin and file gplugin in folder Build.Win/All Plugins

To build everything that can be built on Windows:

make -f scripts/Makefile.gid all

Then you need to copy all the plugins to the 'Build.Win' folder with: 

make -f scripts/Makefile.gid bundle.win

Cleaning

If you want to reset the build process and start again then you can issue a 'clean' command. To do this type 'make -f scripts/Makefile.gid clean'. Example: 

make -f scripts/Makefile.gid clean

Sometimes you need to make clean a particular build. To do this type 'make -f scripts/Makefile.gid' followed by something like 'html5' then '.clean'. Example: 

make -f scripts/Makefile.gid html5.clean











Troubleshooting:

MSYS2 on Windows 10 - cannot update the package database using pacman #2589 Sprinkles44 opened this issue on 28 Jul 2021 · 7 comments MSYS2 on Windows 10 - cannot update the package database using pacman #2589 Sprinkles44 opened this issue on 28 Jul 2021 · 7 comments Comments


Sprinkles44 commented on 28 Jul 2021 I tried to update my MSYS2 on Windows 10 (as per the instructions on https://www.msys2.org/) and I got this error: 

OscarAlvarez@Lambda-Station-OA MINGW64 ~
$ pacman -Syu
error: config file /etc/pacman.d/mirrorlist.mingw could not be read: No such file or directory

So I did a fresh install of MSYS2 on Windows 10. I got the binaries from the MSYS2 website and installed it successfully. Now, I am getting the same error. I went into the /etc/pacman.d/ directory and this is what was inside: 

OscarAlvarez@Lambda-Station-OA MINGW64 /etc/pacman.d
$ ls
gnupg  mirrorlist.clang64  mirrorlist.mingw32  mirrorlist.mingw64  mirrorlist.msys  mirrorlist.ucrt64

I didn't find the file so I created an empty file with the name "mirrorlist.mingw" and ran the update but to no avail. So I just deleted the empty file I created. No other flags like -S, -Su, or -Syy, etc. work and produce the same error. Thank you.

  • OS: Windows 10 Education


Collaborator Biswa96 commented on 28 Jul 2021 There is something wrong in your pc setup. Antivirus software may delete files without any reason. To workaround this situation


Author Sprinkles44 commented on 28 Jul 2021 I only have the Windows 10 antivirus that comes with the OS. I do not have any other antivirus software. I downloaded the file then opened MSYS2 and ran the command. I get the same error: 

OscarAlvarez@Lambda-Station-OA MSYS ~
# pacman -U --overwrite=* pacman-mirrors-20210706-1-any.pkg.tar.zst
error: config file /etc/pacman.d/mirrorlist.mingw could not be read: No such file or directory

It seems to me that I can't run any pacman command. You say to run that command but where do I run it, in MSYS2? If I run it with MSYS2, do I move the downloaded file to a specific directory or will the downloaded file just be detected automatically anywhere on my system? It's in my download folder right now. Thank you.


Collaborator Biswa96 commented on 28 Jul 2021 

error: config file /etc/pacman.d/mirrorlist.mingw could not be read: No such file or directory

Create that empty file as you have said in original post. Then run the pacman -U command. You say to run that command but where do I run it, in MSYS2? Any msys2/mingw window. will the downloaded file just be detected automatically anywhere on my system? You need to specify the full path of that .pkg.tar.zst file in that pacman -U command. Or drag and drop the .pkg.tar.zst file in msys2 window, it will automatically show the full path.


Author Sprinkles44 commented on 30 Jul 2021 

error: config file /etc/pacman.d/mirrorlist.mingw could not be read: No such file or directory

Create that empty file as you have said in original post. Then run the pacman -U command. You say to run that command but where do I run it, in MSYS2? Any msys2/mingw window. will the downloaded file just be detected automatically anywhere on my system? You need to specify the full path of that .pkg.tar.zst file in that pacman -U command. Or drag and drop the .pkg.tar.zst file in msys2 window, it will automatically show the full path. Thank you that seems to have worked (sort of). Now I only get this error: 

OscarAlvarez@Lambda-Station-OA MSYS ~
$ pacman -Syu
:: Synchronizing package databases...
error: failed to synchronize all databases (no servers configured for repository)

I think I have to add to the repository or something like that but I am not too sure. Thank you again for helping.


Collaborator jeremyd2019 commented on 30 Jul 2021 

mv /etc/pacman.d/mirrorlist.mingw.pacnew /etc/pacman.d/mirrorlist.mingw


Author Sprinkles44 commented on 30 Jul 2021 Thank you so much, it works perfectly now.

Retrieved from ‘https://wiki.gideros.rocks/index.php?title=Compiling_Gideros_Luau_Source&oldid=18327’