Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: replaced all whitelist and blacklist occurences by allowlis… #5277

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,9 @@ config/metrics/
API_file.txt
New_API_file.txt

# Ignore Whitelist and Blacklist files
whitelist.json
blacklist.json
# Ignore Allowlist and Denylist files
allowlist.json
denylist.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please retain the old flle names in the gitignore so that they are not accidentally committed from older workspaces in future?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

applied


# Profiling related
terasology.jfr
Expand Down
6 changes: 3 additions & 3 deletions docs/Modding-API.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
Modding API
=================

Terasology's engine uses a whitelisting approach to expose an API for modules using two primary methods and a rarely needed third one:
Terasology's engine uses allowlisting approach to expose an API for modules using two primary methods and a rarely needed third one:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Terasology's engine uses allowlisting approach to expose an API for modules using two primary methods and a rarely needed third one:
Terasology's engine uses an allowlisting approach to expose an API for modules using two primary methods and a rarely needed third one:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or alternatively:

Suggested change
Terasology's engine uses allowlisting approach to expose an API for modules using two primary methods and a rarely needed third one:
Terasology's engine uses allowlisting to expose an API for modules using two primary methods and a rarely needed third one:

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

applied


* Classes or packages marked with the `@API` annotation
* Classes or packages in the basic whitelist defined in `ExternalApiWhitelist.java`
* Classes or packages in the basic allowlist defined in `ExternalApiAllowlist.java`
* Rarely blocks of code in the engine may be hit in a way requiring use of `AccessController.doPrivileged(...)` - usually module authors do not need to worry about this but once in a while it could explain something quirky.

This is to both protect the user's system from malicious code (for instance you cannot use `java.io.File` directly) and to better document what is available. If you attempt to use a class not on the whitelist you'll get log message like:
This is to both protect the user's system from malicious code (for instance you cannot use `java.io.File` directly) and to better document what is available. If you attempt to use a class not on the allowlist you'll get log message like:

`Denied access to class (not allowed with this module's permissions): some.package.and.class`

Expand Down
2 changes: 1 addition & 1 deletion docs/Module-Security.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The game creates new objects and executes methods on them in response to network

Terasology relies on [Gestalt Module Sandboxing](https://github.com/MovingBlocks/gestalt/wiki/Module%20Sandboxing) to protect from these risks of running untrusted JVM code. However, it's up to the application to make sure the sandbox is configured and applied correctly.

* [o.t.engine.core.module.ExternalApiWhitelist](https://github.com/MovingBlocks/Terasology/blob/develop/engine/src/main/java/org/terasology/engine/core/module/ExternalApiWhitelist.java) defines a hardcoded list of allowable packages and classes.
* [o.t.engine.core.module.ExternalApiAllowlist](https://github.com/MovingBlocks/Terasology/blob/develop/engine/src/main/java/org/terasology/engine/core/module/ExternalApiAllowlist.java) defines a hardcoded list of allowable packages and classes.

## ClassLoaders

Expand Down
8 changes: 4 additions & 4 deletions docs/Playing.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,13 +143,13 @@ Alternatively you can run from source and supply parameters for game configurati

This will all become easier as the project and especially the launcher mature further :-)

### Server Whitelist and Blacklist
### Server Allowlist and Denylist

Hosting a server will create a whitelist and a blacklist that can be used to manage who is able to connect to that server.
Hosting a server will create a allowlist and a denylist that can be used to manage who is able to connect to that server.

If the whitelist contains at least one client ID, only the ID(s) on the list will be allowed to connect to the server. All IDs not on the whitelist are effectively blacklisted.
If the allowlist contains at least one client ID, only the ID(s) on the list will be allowed to connect to the server. All IDs not on the allowlist are effectively denylisted.

If the whitelist is empty, any ID not on the blacklist will be able to connect.
If the allowlist is empty, any ID not on the denylist will be able to connect.

Client IDs are added to the lists in JSON format, for example: ["6a5f11f7-4038-4ef0-91ac-86cb957588b1","01264d12-27cf-4699-b8e2-bdc92ac8ef73"]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
import org.terasology.engine.core.Time;
import org.terasology.engine.core.bootstrap.EntitySystemSetupUtil;
import org.terasology.engine.core.modes.loadProcesses.LoadPrefabs;
import org.terasology.engine.core.module.ExternalApiWhitelist;
import org.terasology.engine.core.module.ExternalApiAllowlist;
import org.terasology.engine.core.module.ModuleManager;
import org.terasology.engine.core.subsystem.headless.assets.HeadlessMaterial;
import org.terasology.engine.core.subsystem.headless.assets.HeadlessMesh;
Expand Down Expand Up @@ -264,7 +264,7 @@ protected void setupConfig() {

@Override
protected void setupModuleManager(Set<Name> moduleNames) throws RuntimeException {
TypeRegistry.WHITELISTED_CLASSES = ExternalApiWhitelist.CLASSES.stream().map(Class::getName).collect(Collectors.toSet());
TypeRegistry.WHITELISTED_CLASSES = ExternalApiAllowlist.CLASSES.stream().map(Class::getName).collect(Collectors.toSet());

ModuleManager moduleManager = ModuleManagerFactory.create();
ModuleTypeRegistry typeRegistry = new ModuleTypeRegistry(moduleManager.getEnvironment());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.io.TempDir;
import org.terasology.engine.core.PathManager;
import org.terasology.engine.core.module.ExternalApiWhitelist;
import org.terasology.engine.core.module.ExternalApiAllowlist;
import org.terasology.engine.core.module.ModuleManager;
import org.terasology.engine.testUtil.ModuleManagerFactory;
import org.terasology.reflection.ModuleTypeRegistry;
Expand All @@ -26,8 +26,8 @@ public void before(@TempDir Path tempHome) throws IOException {
PathManager.getInstance().useOverrideHomePath(tempHome);

moduleManager = ModuleManagerFactory.create();
TypeRegistry.WHITELISTED_CLASSES = ExternalApiWhitelist.CLASSES.stream().map(Class::getName).collect(Collectors.toSet());
TypeRegistry.WHITELISTED_PACKAGES = ExternalApiWhitelist.PACKAGES;
TypeRegistry.WHITELISTED_CLASSES = ExternalApiAllowlist.CLASSES.stream().map(Class::getName).collect(Collectors.toSet());
TypeRegistry.WHITELISTED_PACKAGES = ExternalApiAllowlist.PACKAGES;
typeRegistry = new ModuleTypeRegistry(moduleManager.getEnvironment());

setup();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import com.google.common.collect.SortedSetMultimap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.terasology.engine.core.module.ExternalApiWhitelist;
import org.terasology.engine.core.module.ExternalApiAllowlist;
import org.terasology.engine.core.module.ModuleManager;
import org.terasology.engine.testUtil.ModuleManagerFactory;
import org.terasology.gestalt.module.ModuleEnvironment;
Expand Down Expand Up @@ -99,9 +99,9 @@ public static void main(String[] args) throws Exception {
LOGGER.info("Unknown protocol for: {}, came from {}", apiClass, location);
}
}
sortedApi.putAll("external", ExternalApiWhitelist.CLASSES.stream()
sortedApi.putAll("external", ExternalApiAllowlist.CLASSES.stream()
.map(clazz -> clazz.getName() + " (CLASS)").collect(Collectors.toSet()));
sortedApi.putAll("external", ExternalApiWhitelist.PACKAGES.stream()
sortedApi.putAll("external", ExternalApiAllowlist.PACKAGES.stream()
.map(packagee -> packagee + " (PACKAGE)").collect(Collectors.toSet()));

System.out.println("# Modding API:\n");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import com.google.common.collect.Multimaps;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.terasology.engine.core.module.ExternalApiWhitelist;
import org.terasology.engine.core.module.ExternalApiAllowlist;
import org.terasology.engine.core.module.ModuleManager;
import org.terasology.engine.testUtil.ModuleManagerFactory;
import org.terasology.gestalt.module.ModuleEnvironment;
Expand Down Expand Up @@ -100,9 +100,9 @@ static StringBuffer getApi() throws Exception {
logger.error("Unknown protocol for: {} , came from {}", apiClass, location);
}
}
api.putAll(EXTERNAL, ExternalApiWhitelist.CLASSES.stream()
api.putAll(EXTERNAL, ExternalApiAllowlist.CLASSES.stream()
.map(clazz -> clazz.getName() + " (CLASS)").collect(Collectors.toSet()));
api.putAll(EXTERNAL, ExternalApiWhitelist.PACKAGES.stream()
api.putAll(EXTERNAL, ExternalApiAllowlist.PACKAGES.stream()
.map(packagee -> packagee + " (PACKAGE)").collect(Collectors.toSet()));

//Puts the information in the StringBuffer
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import org.junit.jupiter.api.Test;
import org.reflections.Reflections;
import org.terasology.engine.ModuleEnvironmentTest;
import org.terasology.engine.core.module.ExternalApiWhitelist;
import org.terasology.engine.core.module.ExternalApiAllowlist;
import org.terasology.gestalt.entitysystem.component.Component;
import org.terasology.gestalt.naming.Name;

Expand Down Expand Up @@ -49,10 +49,10 @@ public void testModuleTypes() {
// TODO: Re-enable as gradlew check seems to still stall even with the Ignore in place?
@Disabled("Seems to intermittently stall, at least on Win10")
@Test
public void testWhitelistedTypes() {
public void testAllowlistedTypes() {
Set<Class<?>> allTypes = typeRegistry.getSubtypesOf(Object.class);
for (Class<?> whitelisted : ExternalApiWhitelist.CLASSES) {
assumeTrue(allTypes.contains(whitelisted), () -> allTypes.toString() + " should contain " + whitelisted.getName());
for (Class<?> allowlisted : ExternalApiAllowlist.CLASSES) {
assumeTrue(allTypes.contains(allowlisted), () -> allTypes.toString() + " should contain " + allowlisted.getName());
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
import org.terasology.engine.context.internal.ContextImpl;
import org.terasology.engine.core.bootstrap.EnvironmentSwitchHandler;
import org.terasology.engine.core.modes.GameState;
import org.terasology.engine.core.module.ExternalApiWhitelist;
import org.terasology.engine.core.module.ExternalApiAllowlist;
import org.terasology.engine.core.module.ModuleManager;
import org.terasology.engine.core.subsystem.DisplayDevice;
import org.terasology.engine.core.subsystem.EngineSubsystem;
Expand Down Expand Up @@ -327,8 +327,8 @@ private void initManagers() {

changeStatus(TerasologyEngineStatus.INITIALIZING_MODULE_MANAGER);
TypeRegistry.WHITELISTED_CLASSES =
ExternalApiWhitelist.CLASSES.stream().map(Class::getName).collect(Collectors.toSet());
TypeRegistry.WHITELISTED_PACKAGES = ExternalApiWhitelist.PACKAGES;
ExternalApiAllowlist.CLASSES.stream().map(Class::getName).collect(Collectors.toSet());
TypeRegistry.WHITELISTED_PACKAGES = ExternalApiAllowlist.PACKAGES;

ModuleManager moduleManager = new ModuleManager(rootContext.get(Config.class),
classesOnClasspathsToAddToEngine);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import java.lang.reflect.InvocationTargetException;
import java.util.Set;

public final class ExternalApiWhitelist {
public final class ExternalApiAllowlist {
private static final Set<String> NUI_PACKAGES = new ImmutableSet.Builder<String>()
.add("org.terasology.input")
.add("org.terasology.input.device")
Expand Down Expand Up @@ -188,6 +188,6 @@ public final class ExternalApiWhitelist {
.add(org.terasology.reflection.metadata.FieldMetadata.class)
.build();

private ExternalApiWhitelist() {
private ExternalApiAllowlist() {
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -256,8 +256,8 @@ private static ModuleMetadataJsonAdapter newMetadataReader() {

private void setupSandbox() {
PermissionSet permissionSet = permissionProviderFactory.getBasePermissionSet();
ExternalApiWhitelist.CLASSES.forEach(permissionSet::addAPIClass);
ExternalApiWhitelist.PACKAGES.forEach(permissionSet::addAPIPackage);
ExternalApiAllowlist.CLASSES.forEach(permissionSet::addAPIClass);
ExternalApiAllowlist.PACKAGES.forEach(permissionSet::addAPIPackage);

APIScanner apiScanner = new APIScanner(permissionProviderFactory);
for (Module module : registry) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

/**
* This class provides the methods needed to determine if a client is allowed to connect or not,
* based on the blacklist and whitelist files.
* based on the denylist and allowlist files.
*/

public class ServerConnectListManager {
Expand All @@ -28,14 +28,14 @@ public class ServerConnectListManager {
private static final Gson GSON = new Gson();

private Context context;
private Set<String> blacklistedIDs;
private Set<String> whitelistedIDs;
private final Path blacklistPath;
private final Path whitelistPath;
private Set<String> denylistedIDs;
private Set<String> allowlistedIDs;
private final Path denylistPath;
private final Path allowlistPath;

public ServerConnectListManager(Context context) {
blacklistPath = PathManager.getInstance().getHomePath().resolve("blacklist.json");
whitelistPath = PathManager.getInstance().getHomePath().resolve("whitelist.json");
denylistPath = PathManager.getInstance().getHomePath().resolve("denylist.json");
allowlistPath = PathManager.getInstance().getHomePath().resolve("allowlist.json");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is likely to a breaking change otherwise, can we continue to check for the old file names if the newer ones do not exist?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

applied

this.context = context;
loadLists();
}
Expand All @@ -44,29 +44,29 @@ public ServerConnectListManager(Context context) {
private void loadLists() {
try {
if (createFiles()) {
blacklistedIDs = GSON.fromJson(Files.newBufferedReader(blacklistPath), Set.class);
whitelistedIDs = GSON.fromJson(Files.newBufferedReader(whitelistPath), Set.class);
if (blacklistedIDs == null) {
blacklistedIDs = new HashSet<>();
denylistedIDs = GSON.fromJson(Files.newBufferedReader(denylistPath), Set.class);
allowlistedIDs = GSON.fromJson(Files.newBufferedReader(allowlistPath), Set.class);
if (denylistedIDs == null) {
denylistedIDs = new HashSet<>();
}
if (whitelistedIDs == null) {
whitelistedIDs = new HashSet<>();
if (allowlistedIDs == null) {
allowlistedIDs = new HashSet<>();
}
}
} catch (IOException e) {
logger.error("Whitelist or blacklist files not found:", e);
logger.error("Allowlist or denylist files not found:", e);
}
}

private void saveLists() {
try {
if (createFiles()) {
Writer blacklistWriter = Files.newBufferedWriter(blacklistPath);
Writer whitelistWriter = Files.newBufferedWriter(whitelistPath);
blacklistWriter.write(GSON.toJson(blacklistedIDs));
whitelistWriter.write(GSON.toJson(whitelistedIDs));
blacklistWriter.close();
whitelistWriter.close();
Writer denylistWriter = Files.newBufferedWriter(denylistPath);
Writer allowlistWriter = Files.newBufferedWriter(allowlistPath);
denylistWriter.write(GSON.toJson(denylistedIDs));
allowlistWriter.write(GSON.toJson(allowlistedIDs));
denylistWriter.close();
allowlistWriter.close();
}
} catch (IOException e) {
e.printStackTrace();
Expand All @@ -78,63 +78,63 @@ private boolean createFiles() throws IOException {
if (display == null || !display.isHeadless()) {
return false;
}
if (!Files.exists(blacklistPath)) {
Files.createFile(blacklistPath);
if (!Files.exists(denylistPath)) {
Files.createFile(denylistPath);
}
if (!Files.exists(whitelistPath)) {
Files.createFile(whitelistPath);
if (!Files.exists(allowlistPath)) {
Files.createFile(allowlistPath);
}
return true;
}

public String getErrorMessage(String clientID) {
if (isClientBlacklisted(clientID)) {
return "client on blacklist";
if (isClientDenylisted(clientID)) {
return "client on denylist";
}
if (!isClientWhitelisted(clientID)) {
return "client not on whitelist";
if (!isClientAllowlisted(clientID)) {
return "client not on allowlist";
}
return null;
}

@SuppressWarnings("BooleanMethodIsAlwaysInverted")
public boolean isClientAllowedToConnect(String clientID) {
return !isClientBlacklisted(clientID) && isClientWhitelisted(clientID);
return !isClientDenylisted(clientID) && isClientAllowlisted(clientID);
}

public void addToWhitelist(String clientID) {
whitelistedIDs.add(clientID);
public void addToAllowlist(String clientID) {
allowlistedIDs.add(clientID);
saveLists();
}

public void removeFromWhitelist(String clientID) {
whitelistedIDs.remove(clientID);
public void removeFromAllowlist(String clientID) {
allowlistedIDs.remove(clientID);
saveLists();
}

public Set getWhitelist() {
return Collections.unmodifiableSet(whitelistedIDs);
public Set getAllowlist() {
return Collections.unmodifiableSet(allowlistedIDs);
}

public void addToBlacklist(String clientID) {
blacklistedIDs.add(clientID);
public void addToDenylist(String clientID) {
denylistedIDs.add(clientID);
saveLists();
}

public void removeFromBlacklist(String clientID) {
blacklistedIDs.remove(clientID);
public void removeFromDenylist(String clientID) {
denylistedIDs.remove(clientID);
saveLists();
}

public Set getBlacklist() {
return Collections.unmodifiableSet(blacklistedIDs);
public Set getDenylist() {
return Collections.unmodifiableSet(denylistedIDs);
}

private boolean isClientBlacklisted(String clientID) {
return blacklistedIDs != null && blacklistedIDs.contains(clientID);
private boolean isClientDenylisted(String clientID) {
return denylistedIDs != null && denylistedIDs.contains(clientID);
}

private boolean isClientWhitelisted(String clientID) {
return whitelistedIDs == null || whitelistedIDs.isEmpty() || whitelistedIDs.contains(clientID);
private boolean isClientAllowlisted(String clientID) {
return allowlistedIDs == null || allowlistedIDs.isEmpty() || allowlistedIDs.contains(clientID);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -149,20 +149,20 @@ public void onOpened() {

private void loadGame(GameInfo item) {
if (isLoadingAsServer()) {
Path blacklistPath = PathManager.getInstance().getHomePath().resolve("blacklist.json");
Path whitelistPath = PathManager.getInstance().getHomePath().resolve("whitelist.json");
if (!Files.exists(blacklistPath)) {
Path denylistPath = PathManager.getInstance().getHomePath().resolve("denylist.json");
Path allowlistPath = PathManager.getInstance().getHomePath().resolve("allowlist.json");
if (!Files.exists(denylistPath)) {
BenjaminAmos marked this conversation as resolved.
Show resolved Hide resolved
try {
Files.createFile(blacklistPath);
Files.createFile(denylistPath);
} catch (IOException e) {
logger.error("IO Exception on blacklist generation", e);
logger.error("IO Exception on denylist generation", e);
}
}
if (!Files.exists(whitelistPath)) {
if (!Files.exists(allowlistPath)) {
try {
Files.createFile(whitelistPath);
Files.createFile(allowlistPath);
} catch (IOException e) {
logger.error("IO Exception on whitelist generation", e);
logger.error("IO Exception on allowlist generation", e);
}
}
}
Expand Down
Loading