When attempting to build large modules (many Action Blocks within a Procedure Step), getting error in the Gen Build Tool of:

java.lang.OutOfMemoryError: Java heap space

Release : 8.6

Component : CA Gen Run Time, Distributed, BuildTool

Within the Gen Build Tool under the 'Tools' menu item select 'Profile Manager' and within the 'Java' tab, change the token OPT.JAVAFLAGS' value to something higher. 

For example:  Change  this token value  From:  -Xmx256m    To:   -Xmx512m  or higher and retry the build process. 

In addition, if generating the module(s) (Procedure Step and called Action Blocks) with trace, may want to try the generation and subsequent build without trace as well.