前言

准备

源码分析

1. manifest

2. agent分析

3. agent卸载逻辑

总结

前言

笔者在很早前写了(231条消息) OpenRASP Java应用自我保护使用_fenglllle的博客-CSDN博客

实际上很多商业版的rasp工具都是基于OpenRASP的灵感来的，主要就是对核心的Java类通过Javaagent技术，对特定的方法注入字节码，做参数验证。核心技术就是Javaagent，那么分析OpenRASP的agent实现原理，即可明白主流的rasp实现逻辑。在OpenRASP上优化部分实现逻辑就可以形成一个商业产品。

准备

准备百度的admin端，执行./rasp-cloud -d

登录后拿到

appid、appsecret

在Java的靶机上配置agent和agent使用的参数

注意百度自己的一键注入代码jar，注入的agent在其他所有agent之前，agent的加载是有顺序的。

源码分析

1. manifest

agent是mainfest定义的入口，可以在pom插件和java包内部看到

pom插件

jar包

2. agent分析

以jvm参数方式为例，动态注入也差不多，rasp一般使用都是jvm参数启动时注入

com.baidu.openrasp.Agent

    public static void premain(String agentArg, Instrumentation inst) {
        init(START_MODE_NORMAL, START_ACTION_INSTALL, inst);
    }

    public static synchronized void init(String mode, String action, Instrumentation inst) {
        try {
            //添加jar文件到jdk的跟路径下，优先加载
            JarFileHelper.addJarToBootstrap(inst);
            //读取一些agent的数据
            readVersion();
            //核心代码
            ModuleLoader.load(mode, action, inst);
        } catch (Throwable e) {
            System.err.println("[OpenRASP] Failed to initialize, will continue without security protection.");
            e.printStackTrace();
        }
    }

ModuleLoader.load(mode, action, inst);

    /**
     * 加载所有 RASP 模块
     *
     * @param mode 启动模式
     * @param inst {@link java.lang.instrument.Instrumentation}
     */
    public static synchronized void load(String mode, String action, Instrumentation inst) throws Throwable {
        if (Module.START_ACTION_INSTALL.equals(action)) {
            if (instance == null) {
                try {
                    //安装
                    instance = new ModuleLoader(mode, inst);
                } catch (Throwable t) {
                    instance = null;
                    throw t;
                }
            } else {
                System.out.println("[OpenRASP] The OpenRASP has bean initialized and cannot be initialized again");
            }
        } else if (Module.START_ACTION_UNINSTALL.equals(action)) {
            //卸载
            release(mode);
        } else {
            throw new IllegalStateException("[OpenRASP] Can not support the action: " + action);
        }
    }

看看new ModuleLoader逻辑

    /**
     * 构造所有模块
     *
     * @param mode 启动模式
     * @param inst {@link java.lang.instrument.Instrumentation}
     */
    private ModuleLoader(String mode, Instrumentation inst) throws Throwable {
        // JBoss参数，实际上就是设置一些系统变量
        if (Module.START_MODE_NORMAL == mode) {
            setStartupOptionForJboss();
        }
        //构造对象，加载另外的jar engine.jar 初始化com.baidu.openrasp.EngineBoot
        engineContainer = new ModuleContainer(ENGINE_JAR);
        //核心逻辑
        engineContainer.start(mode, inst);
    }

new ModuleContainer(ENGINE_JAR);

    public ModuleContainer(String jarName) throws Throwable {
        try {
            //engine jar，openrasp有2个jar，单独加载，方便卸载，agent常用手段
            File originFile = new File(baseDirectory + File.separator + jarName);
            JarFile jarFile = new JarFile(originFile);
            Attributes attributes = jarFile.getManifest().getMainAttributes();
            jarFile.close();
            this.moduleName = attributes.getValue("Rasp-Module-Name");
            // com.baidu.openrasp.EngineBoot
            String moduleEnterClassName = attributes.getValue("Rasp-Module-Class");
            //用classloader加载jar
            if (moduleName != null && moduleEnterClassName != null
                    && !moduleName.equals("") && !moduleEnterClassName.equals("")) {
                Class moduleClass;
                if (ClassLoader.getSystemClassLoader() instanceof URLClassLoader) {
                    Method method = Class.forName("java.net.URLClassLoader").getDeclaredMethod("addURL", URL.class);
                    method.setAccessible(true);
                    method.invoke(moduleClassLoader, originFile.toURI().toURL());
                    method.invoke(ClassLoader.getSystemClassLoader(), originFile.toURI().toURL());
                    moduleClass = moduleClassLoader.loadClass(moduleEnterClassName);
                    //com.baidu.openrasp.EngineBoot对象
                    module = (Module) moduleClass.newInstance();
                } else if (ModuleLoader.isCustomClassloader()) {
                    moduleClassLoader = ClassLoader.getSystemClassLoader();
                    Method method = moduleClassLoader.getClass().getDeclaredMethod("appendToClassPathForInstrumentation", String.class);
                    method.setAccessible(true);
                    try {
                        method.invoke(moduleClassLoader, originFile.getCanonicalPath());
                    } catch (Exception e) {
                        method.invoke(moduleClassLoader, originFile.getAbsolutePath());
                    }
                    moduleClass = moduleClassLoader.loadClass(moduleEnterClassName);
                    //初始化com.baidu.openrasp.EngineBoot
                    module = (Module) moduleClass.newInstance();
                } else {
                    throw new Exception("[OpenRASP] Failed to initialize module jar: " + jarName);
                }
            }
        } catch (Throwable t) {
            System.err.println("[OpenRASP] Failed to initialize module jar: " + jarName);
            throw t;
        }
    }

engineContainer.start(mode, inst);

    public void start(String mode, Instrumentation inst) throws Exception {
        //帅气的时刻，打印logo
        System.out.println("\n\n" +
                "   ____                   ____  ___   _____ ____ \n" +
                "  / __ \\____  ___  ____  / __ \\/   | / ___// __ \\\n" +
                " / / / / __ \\/ _ \\/ __ \\/ /_/ / /| | \\__ \\/ /_/ /\n" +
                "/ /_/ / /_/ /  __/ / / / _, _/ ___ |___/ / ____/ \n" +
                "\\____/ .___/\\___/_/ /_/_/ |_/_/  |_/____/_/      \n" +
                "    /_/                                          \n\n");
        try {
            //载入v8，调用c，实际上可以使用火狐引擎替代，百度的官方文档的图就是rhino
            Loader.load();
        } catch (Exception e) {
            System.out.println("[OpenRASP] Failed to load native library, please refer to https://rasp.baidu.com/doc/install/software.html#faq-v8-load for possible solutions.");
            e.printStackTrace();
            return;
        }
        if (!loadConfig()) {
            return;
        }
        //缓存rasp的build信息
        Agent.readVersion();
        BuildRASPModel.initRaspInfo(Agent.projectVersion, Agent.buildTime, Agent.gitCommit);
        // 初始化插件系统 调用v8，支持动态更新插件和js
        if (!JS.Initialize()) {
            return;
        }
        //核心代码，加入检查点，就是哪些类的哪些方法需要check
        CheckerManager.init();
        //字节码替换
        initTransformer(inst);
        if (CloudUtils.checkCloudControlEnter()) {
            CrashReporter.install(Config.getConfig().getCloudAddress() + "/v1/agent/crash/report",
                    Config.getConfig().getCloudAppId(), Config.getConfig().getCloudAppSecret(),
                    CloudCacheModel.getInstance().getRaspId());
        }
        deleteTmpDir();
        String message = "[OpenRASP] Engine Initialized [" + Agent.projectVersion + " (build: GitCommit="
                + Agent.gitCommit + " date=" + Agent.buildTime + ")]";
        System.out.println(message);
        Logger.getLogger(EngineBoot.class.getName()).info(message);
    }

loadConfig()

    private boolean loadConfig() throws Exception {
        //处理日志相关
        LogConfig.ConfigFileAppender();
        //单机模式下动态添加获取删除syslog
        if (!CloudUtils.checkCloudControlEnter()) {
            LogConfig.syslogManager();
        } else {
            System.out.println("[OpenRASP] RASP ID: " + CloudCacheModel.getInstance().getRaspId());
        }
        return true;
    }

    private void init() {
        this.configFileDir = baseDirectory + File.separator + CONFIG_DIR_NAME;
        String configFilePath = this.configFileDir + File.separator + CONFIG_FILE_NAME;
        try {
            //载入靶机的配置文件，里面配置了appid appsecret等
            loadConfigFromFile(new File(configFilePath), true);
            if (!getCloudSwitch()) {
                try {
                    FileScanMonitor.addMonitor(
                            baseDirectory, instance);
                } catch (JNotifyException e) {
                    throw new ConfigLoadException("add listener on " + baseDirectory + " failed because:" + e.getMessage());
                }
                //支持动态刷新文件
                addConfigFileMonitor();
            }
        } catch (FileNotFoundException e) {
            handleException("Could not find openrasp.yml, using default settings: " + e.getMessage(), e);
        } catch (JNotifyException e) {
            handleException("add listener on " + configFileDir + " failed because:" + e.getMessage(), e);
        } catch (Exception e) {
            handleException("cannot load properties file: " + e.getMessage(), e);
        }
        String configValidMsg = checkMajorConfig();
        if (configValidMsg != null) {
            LogTool.error(ErrorType.CONFIG_ERROR, configValidMsg);
            throw new ConfigLoadException(configValidMsg);
        }
    }

JS.Initialize()，JS引擎初始化，js文件监听

    public synchronized static boolean Initialize() {
        try {
            //v8初始化检查
            if (!V8.Initialize()) {
                throw new Exception("[OpenRASP] Failed to initialize V8 worker threads");
            }
            //log
            V8.SetLogger(new com.baidu.openrasp.v8.Logger() {
                @Override
                public void log(String msg) {
                    pluginLog(msg);
                }
            });
            //
            V8.SetStackGetter(new com.baidu.openrasp.v8.StackGetter() {
                @Override
                public byte[] get() {
                    try {
                        ByteArrayOutputStream stack = new ByteArrayOutputStream();
                        JsonStream.serialize(StackTrace.getParamStackTraceArray(), stack);
                        stack.write(0);
                        return stack.getByteArray();
                    } catch (Exception e) {
                        return null;
                    }
                }
            });
            Context.setKeys();
            //是否云端控制，比如js更新，插件更新
            if (!CloudUtils.checkCloudControlEnter()) {
                //更新插件
                UpdatePlugin();
                //js文件监听器
                InitFileWatcher();
            }
            return true;
        } catch (Exception e) {
            e.printStackTrace();
            LOGGER.error(e);
            return false;
        }
    }


    public synchronized static boolean UpdatePlugin(List<String[]> scripts) {
        boolean rst = V8.CreateSnapshot(pluginConfig, scripts.toArray(), BuildRASPModel.getRaspVersion());
        if (rst) {
            try {
                //执行js
                String jsonString = V8.ExecuteScript("JSON.stringify(RASP.algorithmConfig || {})",
                        "get-algorithm-config.js");
                Config.getConfig().setConfig(ConfigItem.ALGORITHM_CONFIG, jsonString, true);
            } catch (Exception e) {
                LogTool.error(ErrorType.PLUGIN_ERROR, e.getMessage(), e);
            }
            Config.commonLRUCache.clear();
        }
        return rst;
    }

CheckerManager.init();

    public synchronized static void init() throws Exception {
        for (Type type : Type.values()) {
            checkers.put(type, type.checker);
        }
    }

检查的类是写在代码里，意味着新增类需要重启，不过一般是增加参数更新规则，新增类频率比较低

自定义转换器，retransform

    private void initTransformer(Instrumentation inst) throws UnmodifiableClassException {
        transformer = new CustomClassTransformer(inst);
        transformer.retransform();
    }

这里很关键

    public CustomClassTransformer(Instrumentation inst) {
        this.inst = inst;
        inst.addTransformer(this, true);
        //定义了哪些类需要类替换
        addAnnotationHook();
    }

addAnnotationHook();

    private void addAnnotationHook() {
        //com.baidu.openrasp.hook HookAnnotation注解的类
        Set<Class> classesSet = AnnotationScanner.getClassWithAnnotation(SCAN_ANNOTATION_PACKAGE, HookAnnotation.class);
        for (Class clazz : classesSet) {
            try {
                Object object = clazz.newInstance();
                if (object instanceof AbstractClassHook) {
                    addHook((AbstractClassHook) object, clazz.getName());
                }
            } catch (Exception e) {
                LogTool.error(ErrorType.HOOK_ERROR, "add hook failed: " + e.getMessage(), e);
            }
        }
    }

addHook((AbstractClassHook) object, clazz.getName());//实际上是缓存需要替换的类，还原的时候可以使用

关键还是transform方法

    public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined,
                            ProtectionDomain domain, byte[] classfileBuffer) throws IllegalClassFormatException {
        if (loader != null) {
            DependencyFinder.addJarPath(domain);
        }
        if (loader != null && jspClassLoaderNames.contains(loader.getClass().getName())) {
            jspClassLoaderCache.put(className.replace("/", "."), new SoftReference<ClassLoader>(loader));
        }
        //关键点，使用javassist字节码增强
        for (final AbstractClassHook hook : hooks) {
            if (hook.isClassMatched(className)) {
                CtClass ctClass = null;
                try {
                    ClassPool classPool = new ClassPool();
                    addLoader(classPool, loader);
                    ctClass = classPool.makeClass(new ByteArrayInputStream(classfileBuffer));
                    if (loader == null) {
                        hook.setLoadedByBootstrapLoader(true);
                    }
                    //字节码增强了，包括Tomcat，jdk等类
                    classfileBuffer = hook.transformClass(ctClass);
                    if (classfileBuffer != null) {
                        checkNecessaryHookType(hook.getType());
                    }
                } catch (IOException e) {
                    e.printStackTrace();
                } finally {
                    if (ctClass != null) {
                        ctClass.detach();
                    }
                }
            }
        }
        serverDetector.detectServer(className, loader, domain);
        return classfileBuffer;
    }

定义了一系列的埋点类替换

以com.baidu.openrasp.hook.file.FileHook为例

增加了对File的list方法调用前，对参数进行检查

    protected void hookMethod(CtClass ctClass) throws IOException, CannotCompileException, NotFoundException {
        String src = getInvokeStaticSrc(FileHook.class, "checkListFiles", "$0", File.class);
        insertBefore(ctClass, "list", "()[Ljava/lang/String;", src);
    }

openrasp的架构图和开源代码不对应，开源代码用的v8引擎，这里画的图是Rhino，是原生Java代码，不需要JNI

3. agent卸载逻辑

release(mode)

    public static synchronized void release(String mode) {
        try {
            if (engineContainer != null) {
                System.out.println("[OpenRASP] Start to release OpenRASP");

                engineContainer.release(mode);
                engineContainer = null;
            } else {
                System.out.println("[OpenRASP] Engine is initialized, skipped");
            }
        } catch (Throwable throwable) {
            // ignore
        }
    }

engineContainer.release(mode);

    public void release(String mode) {
        //任务线程停止
        CloudManager.stop();
        //CPU 监控线程停掉
        CpuMonitorManager.release();
        if (transformer != null) {
            transformer.release(); //类替换还原
        }
        JS.Dispose(); //JS检查还原
        CheckerManager.release(); //java 检查还原
        String message = "[OpenRASP] Engine Released [" + Agent.projectVersion + " (build: GitCommit="
                + Agent.gitCommit + " date=" + Agent.buildTime + ")]";
        System.out.println(message);
    }

    public synchronized static void Dispose() {
        if (watchId != null) {
            boolean oldValue = HookHandler.enableHook.getAndSet(false);
            FileScanMonitor.removeMonitor(watchId);
            watchId = null;
            HookHandler.enableHook.set(oldValue);
        }
    }