专属领域论文订阅
关注{晓理紫|小李子}，每日更新论文，如感兴趣，请转发给有需要的同学，谢谢支持
如果你感觉对你有所帮助，请关注我，每日准时为你推送最新论文。
分类:
- 大语言模型LLM
- 视觉模型VLM
- 扩散模型
- 视觉语言导航VLN
- 强化学习 RL
- 模仿学习 IL
- 机器人
- 开放词汇，检测分割
== RLHF ==
标题: Resource Allocation in Vehicular Networks Based on Federated Multi-Agent Reinforcement Learning
作者: Jiaming Yu, Shaochuan Wu, Le Liang
PubTime: 2023-10
Downlink: https://ieeexplore.ieee.org/document/10419509/
Journal: 2023 IEEE 23rd International Conference on Communication Technology (ICCT)
中文摘要: 本文提出了一种基于联邦多智能体深度强化学习（Fed-MARL）的分布式资源分配方案，以解决车载网络中的信道分配和功率控制问题。我们通过利用深度强化学习和联合学习来解决公式化的资源优化问题，以满足车辆到基础设施（V2I）和车辆到车辆（V2V）链路的不同服务质量要求。具体来说，我们建议用联合学习来增强传统的强化学习方法，包括深度Q网络和近似策略优化，以获得两种有效的基于Fed-MARL的车载网络资源分配算法。仿真结果表明，与没有联邦学习辅助的其他基线相比，我们提出的资源分配方案在V2I链路的总容量和V2V链路的有效载荷交付率方面同时表现出优势。
摘要: In this paper, we propose a distributed resource allocation scheme based on federated multi-agent deep reinforcement learning (Fed-MARL) to address the channel allocation and power control problem in vehicular networks. We tackle the formulated resource optimization problem by taking advantage of deep reinforcement learning and federated learning, to satisfy the different quality-of-service requirements for vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) links. Specifically, we propose to enhance traditional reinforcement learning methods, including both the deep Q network and proximal policy optimization, with federated learning, to obtain two efficient Fed-MARL-based resource allocation algorithms for vehicular networks. Simulation results show that our proposed resource allocation schemes exhibit superiority in both the total capacity of V2I links and the payload delivery rate of V2V links simultaneously, compared to other baselines without federated learning assistance.
标题: Game confrontation model of USV based on multi-agent reinforcement learning
作者: Kai Liu, Xiao Wang, ChunLei Wang
PubTime: 2023-08
Downlink: https://ieeexplore.ieee.org/document/10263075/
Journal: 2023 IEEE 9th International Conference on Cloud Computing and Intelligent Systems (CCIS)
中文摘要: 基于深度强化学习在解决多智能体博弈问题中的优异性能，本文提出了一种多无人水面飞行器（多USV）对抗博弈模型。该模型旨在解决多个USV之间的博弈问题，通过深度强化学习学习USV的策略和行为，完成对抗博弈。我们在仿真环境中进行了实验，并将该模型应用于多USV的合作博弈对抗任务中。结果表明，该模型能够学习敌方车辆联合攻击的策略，具有很好的实用功能和研究价值。针对箱式存储环境下多自主移动机器人（AMR）的调度问题，传统的动态规划（DP）算法在求解可行路径时效率低。为了解决这一问题，本文建立了以时间最优化为目标的强化学习（RL）算法模型，用于提高多AMR同时调度的路径规划速度。此外，结合深度学习（DL）算法的优势，使用深度强化学习（DRL）算法，有效缩短了高维复杂工作条件下RL算法模型训练的收敛时间。通过在仿真平台上比较DP、RL和DRL算法模型，验证了DRL方法的有效性。
摘要: Based on the excellent performance of deep reinforcement learning in solving multi-agent game problems, this paper proposes a multi-unmanned surface vehicle (multi-USV) confrontation game model. The model aims to solve the game problem among multi-USV, learn the strategies and behaviors of USV through deep reinforcement learning, and complete the confrontation game. We carry out experiments in the simulation environment, and apply the model to the cooperative game confrontation task of multiple USV. The results show that the model can learn the strategy of joint attack on enemy vehicles, which has a good practical function great and research value.
标题: Deep Reinforcement Learning-based Multi-AMR Path Planning Algorithm
作者: Zixiang Shen, Yunsen Duan, Yongzheng Cong
PubTime: 2023-07
Downlink: https://ieeexplore.ieee.org/document/10240541/
Journal: 2023 42nd Chinese Control Conference (CCC)
摘要: Aiming at the scheduling problem of multi-autonomous mobile robots (AMR) in the box storage environment, the traditional dynamic programming (DP) algorithm has the disadvantage of low efficiency in solving the feasible path. To solve this problem, this paper establishes a reinforcement learning (RL) algorithm model with the goal of time optimization, which is used to improve the speed of path planning for multi-AMR simultaneous scheduling. In addition, combined with the advantages of the deep learning (DL) algorithm, the deep reinforcement learning (DRL) algorithm is used to effectively shorten the convergence time of the RL algorithm model training under high-dimensional and complex working conditions. The effectiveness of the DRL method is verified by comparing DP, RL, and DRL algorithm models in the simulation platform.
标题: Deep Reinforcement Learning for Multiple Agents in a Decentralized Architecture: A Case Study in the Telecommunication Domain
作者: Hongyi Zhang, Jingya Li, Zhiqiang Qi
PubTime: 2023-03
Downlink: https://ieeexplore.ieee.org/document/10092680/
Journal: 2023 IEEE 20th International Conference on Software Architecture Companion (ICSA-C)
中文摘要: 深度强化学习近年来取得了重大发展，目前不仅应用于模拟器和游戏，还应用于嵌入式系统。然而，当在现实世界环境中实施时，强化学习经常被证明是不稳定的，并且不能适应现实情况，特别是当指导大量代理时。在本文中，我们开发了一种用于强化学习的分散式架构，以允许多个代理在它们自己的同类设备上学习最优控制策略，但环境不同。对于这样的多主体，传统的集中式学习算法通常需要昂贵或耗时的努力来开发最佳调节策略，并且不能扩展到大规模系统。为了解决这个问题，我们提出了一种针对每个单独设备的分散强化学习算法（DecRL）和信息交换方案，其中每个代理基于局部模型训练与其他代理共享单独的学习经验和信息。我们将该算法整合到所提出的协作架构中的每个代理中，并在紧急情况下在电信域中进行验证，其中宏基站（BS）因自然灾害而中断，部署三架携带BS（UAV-BS）的无人机为灾区的关键任务（MC）用户提供临时覆盖。基于这些发现，我们表明所提出的分散强化学习算法可以成功地支持多智能体学习，同时可以进一步提高学习速度和服务质量。
摘要: Deep reinforcement learning has made significant development in recent years, and it is currently applied not only in simulators and games but also in embedded systems. However, when implemented in a real-world context, reinforcement learning is frequently shown to be unstable and incapable of adapting to realistic situations, particularly when directing a large number of agents. In this paper, we develop a decentralized architecture for reinforcement learning to allow multiple agents to learn optimal control policies on their own devices of the same kind but in varied environments. For such multiple agents, the traditional centralized learning algorithm usually requires a costly or time-consuming effort to develop the best-regulating policy and is incapable of scaling to a large-scale system. To address this issue, we propose a decentralized reinforcement learning algorithm (DecRL) and information exchange scheme for each individual device, in which each agent shares the individual learning experience and information with other agents based on local model training. We incorporate the algorithm into each agent in the proposed collaborative architecture and validate it in the telecommunication domain under emergency conditions, in which a macro base station (BS) is broken due to a natural disaster, and three unmanned aerial vehicles carrying BSs (UAV-BSs) are deployed to provide temporary coverage for mission-critical (MC) users in the disaster area. Based on the findings, we show that the proposed decentralized reinforcement learning algorithm can successfully support multi-agent learning, while the learning speed and service quality can be further enhanced.
标题: A Study of Model Based and Model Free Offline Reinforcement Learning
作者: Indu Shukla, Haley R. Dozier, Althea C. Henslee
PubTime: 2022-12
Downlink: https://ieeexplore.ieee.org/document/10216717/
Journal: 2022 International Conference on Computational Science and Computational Intelligence (CSCI)
中文摘要: 强化学习（RL）已经受到了相当大的关注，用于构建具有经过训练的代理的自治系统，这些代理与环境交互以学习最佳行为。RL需要一个基本的在线学习范式，这是广泛采用RL扩展到许多现实世界场景的最大障碍之一。在这项研究中，我们探索了数据收集的方法，并在经典的OpenAIGym Cart-Pole环境中应用了离线无模型和基于模型的RL方法。在这种方法中，离线算法从先前收集的数据集中找到一个好的策略。
摘要: Reinforcement learning (RL) has received considerable attention for building autonomous systems with trained agents that interact with environments to learn optimal behavior. The RL requires a fundamentally online learning paradigm, one of the biggest obstacles for the widespread adoption of RL in scaling to many real-world scenarios. In this study we have explored the method of data collection and have applied offline model-free and model-based RL methods on classical OpenAIGym Cart-Pole environment. In this approach offline algorithms find a good policy from previously collected dataset.
标题: A game theoretic approach to curriculum reinforcement learning
作者: Michalis Smyrnakis, Lan Hoang
PubTime: 2022-11
Downlink: https://ieeexplore.ieee.org/document/10098091/
Journal: 2022 IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI)
中文摘要: 当前的强化学习自动化课程通过更新环境来实现持续学习。更新通常被视为优化问题——教师代理更新环境以优化学生的学习。这项工作提出了一个使用博弈论公式的问题的替代框架。学习由领导者——追随者合作博弈定义。该公式为多agent课程学习提供了一种方法，改进了agent学习并提供了更多的博弈均衡见解。我们观察到，在这个框架下，与强化学习代理基线相比，代理收敛得更快，以执行期望的结果。
摘要: Current reinforcement learning automated curricu-lum approaches continual learning by updating the environment. The update is often treated as an optimisation problem - with the teacher agent updating the environment to optimise the student’s learning. This work proposes an alternative framing of the problem using a game-theoretic formulation. The learning is defined by a leader - follower cooperative game. This formulation provides an approach for multi-agent curriculum learning that improves agent learning and provides more game equilibrium insights. We observed that under this framework, the agents converge faster to perform on the desired outcomes, compared to the reinforcement learning agent baseline.
== Imitation Learning ==
标题: Adaptive Guidance by Multi-Agent Systems Combining Personality Imitation and Independent Learning
作者: Kotaro Kobayashi, Ryo Watanabe, Hiroshi Igarashi
PubTime: 2023-12
Downlink: https://ieeexplore.ieee.org/document/10354570/
Journal: 2023 IEEE International Conference on Robotics and Biomimetics (ROBIO)
中文摘要: 牧羊系统是使用一个或多个机器人将逃跑的目标引导到目的地。据说该系统不仅可以用于引导羊群，还可以用于各种现实世界的应用，如通过引导火焰来扑灭森林火灾，引导人类。因此，一个能够真实世界条件下各种干扰的制导系统被认为是必要的。然而，大多数现有的制导系统关注的是它们如何有效地引导目标。另一方面，也有一些研究旨在通过使用机器学习等方法来改变目标的状态，从而实现自适应行为。但是这些研究还不能指导多次改变状态或以意想不到的方式表现的目标。在这项研究中，我们提出了一种算法，可以自适应地引导尚未预先建模的未知目标。目标自适应方法基于社会学习理论，是一种社会认知算法，将模仿机器人行为作为机器人的个性模仿和自主学习。本文的目的不是用某种行为来最优地引导目标。目标是在引导行为没有预先建模的目标时达到一定的性能水平。结果表明，通过个性模仿和自主学习，机器人的参数可以调整到目标。未来，我们计划改进独立学习算法以实现更高的制导性能，改进独立学习算法以实现更高的制导性能。
摘要: Shepherding system is used one or more robots to guide an escaping target to a destination. It is said that this system can be applied not only to guide sheep flock, but also to various real-world applications such as extinguishing forest fires by guiding flames, guiding human. Therefore, a guidance system that can real-world conditions with a variety of disturbances is considered necessary. However, most of the existing guidance systems focus on how efficiently they can guide a target. On the other hand, there are also studies that aim for adaptive behavior by using methods such as machine learning to change the target’s state. But these studies have not been able to guide targets that change their states multiple times or behave in unexpected ways. In this study, we propose an algorithm that can adaptively guide unknown targets that have not been modeled in advance. The target adaptation method is based on the social learning theory, which is an algorithm for social cognition, and uses imitation of robot behavior as personality imitation and independent learning by robots. This paper does not aim to optimally guide a target with a certain behavior. The goal is to achieve a certain level of performance in guiding a target whose behavior has not been modeled in advance. The results show that the parameters of the robot are tuned to the target by personality imitation and independent learning. In the future, we plan to improve the independent learning algorithm to achieve higher guidance performance and improve the independent learning algorithm to achieve higher guidance performance.
标题: Development of a Hand Motion Tracking based Teleoperation System for Imitation learning
作者: Jinchul Choi, Heechul Bae, Chan-Won Park
PubTime: 2023-10
Downlink: https://ieeexplore.ieee.org/document/10392729/
Journal: 2023 14th International Conference on Information and Communication Technology Convergence (ICTC)
中文摘要: 模仿学习正在成为机器人获取技能的一种有前途的方法。由于模仿学习提供了一种通过模仿专家行为来学习策略的方法，所以它需要足够数量的复杂的专家行为轨迹。然而，当前的界面，如动觉教学或远程操作，在有效地收集不同的演示数据方面具有显著的局限性。为了解决这个问题，这项工作提出了一种替代的模仿界面，使用手跟踪解决方案简化了演示采集过程，同时促进了人类动作向机器人的转移。实验结果表明，该系统是一种有效的模拟学习方法，不仅能以较低的时延模拟专家演示，而且有助于收集精细的演示数据
摘要: Imitation learning is emerging as one promising approach for robots to acquire skills. Since imitation learning provides a way to learn policies by imitating the behavior of experts, it requires a sufficient amount of sophisticated expert behavior trajectories. However, current interfaces, such as kinesthetic teaching or remote manipulation, have significant limitations in efficiently collecting diverse demonstration data. To address this issue, this work proposes an alternative interface for imitation that simplifies the demonstration acquisition process using a hand tracking solution while facilitating the transfer of human actions to the robot. Performance results show that the proposed system is effective for imitative learning, not only imitating expert demonstrations with low latency, but also helping to collect elaborate demonstration data.
标题: End-To-End Autonomous Vehicle Navigation Using Imitation Learning
作者: M Vijayalakshmi, D Suvitha, P Sivaguru
PubTime: 2023-08
Downlink: https://ieeexplore.ieee.org/document/10249804/
Journal: 2023 12th International Conference on Advanced Computing (ICoAC)
中文摘要: 提出的模型是一种新颖的深度学习模型，结合了端到端和多任务学习方法，用于自动驾驶中的点对点导航。我们的模型同时执行感知和控制任务，以确保安全可靠的导航。该模型的感知模块从RGBD相机获取高维观测数据，并执行语义分割、语义深度云（SDC）映射以及预测交通灯状态和停车标志等任务。编码的特征，连同GPS和速度计信息，然后由控制模块解码，以预测潜在特征空间中的路点。两个代理处理这些输出，并生成一个控制策略，确定适当的转向、油门和制动动作。该模型使用CARLA模拟器进行评估，考虑各种场景，包括正常和敌对情况，以及不同的天气条件，以模拟真实世界的驾驶条件。该模型的性能与最近的模型在驾驶得分方面进行了比较，并对SDC映射和多智能体方法进行了消融研究，以了解它们的影响。即使参数和计算负载较少，该模型也能获得最高的驾驶分数。
摘要: The proposed model is a novel deep learning model that combines end-to-end and multi-task learning approaches for point-to-point navigation in autonomous driving. Our model simultaneously performs perception and control tasks to ensure safe and reliable navigation. The perception module of the model takes high-dimensional observations from an RGBD camera and performs tasks such as semantic segmentation, semantic depth cloud (SDC) mapping, and predicting traffic light states and stop signs. The encoded features, along with GPS and speedometer information, are then decoded by the control module to predict waypoints in a latent feature space. Two agents process these outputs and generate a control policy that determines the appropriate steering, throttle, and brake actions. The model is evaluated using the CARLA simulator, considering various scenarios including normal and adversarial situations, as well as different weather conditions to simulate real-world driving conditions. The model’s performances is compared with recent models in terms of driving score and conduct an ablation study on SDC mapping and multi-agent approaches to understand their impact. The model achieves the highest driving score even with fewer parameters and computational load.
标题: Imitation Learning for Autonomous Driving Cars
作者: Omer Qureshi, Muhammad Nouman Durrani, Syed Ali Raza
PubTime: 2023-02
Downlink: https://ieeexplore.ieee.org/document/10136686/
Journal: 2023 3rd International Conference on Artificial Intelligence (ICAI)
中文摘要: 世界正处于另一场工业革命的中心。但这一次，真正的革命将由全球计算机科学家领导，而不是蒸汽机，他们将永远改变我们与环境互动的方式。一小部分突破性的研究一直在自动驾驶领域进行，以确保乘客的安全和人类的舒适。自动驾驶主要依赖于机器学习的子集，即模仿学习，几十年来一直是研究的主题。自动驾驶的关键问题是预测车辆的转向角。行为克隆是模仿学习的一种形式，它从人类专家的行为中学习。然而，模仿学习有其自身的一系列挑战，在某些条件下表现不佳。在这项研究中，提出了一种新的算法，CNNO，来预测车辆的转向角。它有五个卷积层、两个最大池层、四个全连接层、展平层和一个脱落层。随后将其与CNN-神经回路策略、CNN、ResNet50、VGG16和VGG19架构进行比较。所提出的算法在10、30和50个时期给出了最佳的评估误差结果，在70个时期给出了最佳的训练误差结果。
摘要: The world is the middle of another industrial revolution. But this time, instead of the steam engines, the real revolution will be led by computer scientists across the globe who will forever change the way we interact with our environment. A small subset of groundbreaking research has been going on in the field of autonomous driving to ensure the safety of passengers and human comfort. Autonomous driving, which primarily relies on the subset of machine learning i.e., imitation learning has been a subject of research for several decades now. The critical problem in autonomous driving is predicting the steering angles of the vehicle. Behavior cloning is a form of imitation learning and it learns from the actions of human experts. However, imitation learning has its own set of challenges and performs poorly in certain conditions. In this research a new algorithm is proposed, CNNO, to predict the steering angles of the vehicle. It has five convolution layers, two max pool layers, four fully connected layers, flatten layer, and a drop-out layer. It is subsequently compared against CNN-Neural Circuit Policy, CNN, ResNet50, VGG16, and VGG19 architectures. The proposed algorithm has shown to give the best evaluation error results from epochs 10, 30 & 50 and the best training error in epoch 70.
标题: Learning from Suboptimal Demonstration via Trajectory-Ranked Adversarial Imitation
作者: Luyao Chen, Shaorong Xie, Tao Pang
PubTime: 2022-11
Downlink: https://ieeexplore.ieee.org/document/10098047/
Journal: 2022 IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI)
中文摘要: 通过模仿学习（IL）训练的机器人用于许多任务（例如，自主车辆操纵）。生成对抗性模仿学习（GAIL）假设用于训练的演示集是高质量的。然而，这种演示很难获得，而且成本很高。如果使用非高质量的演示，GAIL相关的方法无法学习有效的策略，因为由该方法训练的代理的性能受到演示者操作的限制。我们的想法是使代理能够从次优演示集中学习比演示者性能更好的策略，次优演示集中包含更容易获得的非高质量演示。受此启发，我们提出了轨迹排序对抗模仿学习（TRAIL）方法。首先，对于演示集的处理，我们引入了一个排序过程，并定义了次优演示的性能相对优势的概念来指定排序顺序。其次，对于模型训练，我们重建GAIL的目标函数并使用经验重放缓冲区，使代理能够从排名次优的演示集中学习隐含特征和排名信息，并拥有超越演示者的能力。实验表明，在Mujoco的任务中，我们的方法可以从次优演示集学习，并且可以获得比基线方法更好的性能。
摘要: Robots trained by Imitation Learning(IL) are used in many tasks(e.g., autonomous vehicle manipulation). Generative Adversarial Imitation Learning (GAIL) assumes that the demonstration set used for training is of high quality. However, such demonstrations are difficult and expensive to obtain. GAIL-related methods fail to learn effective strategies if non-high quality demonstrations are used because the performance of agents trained by this method is limited by the demonstrator’s operations. Our idea is to enable the agent to learn strategy with better performance than the demonstrator from a suboptimal demonstration set, which contains non-high quality demonstrations that are easier to obtain. Inspired by this, we propose the Trajectory-Ranked Adversarial Imitation Learning (TRAIL) method. First, for demonstration set processing, we introduce a ranking process and define the concept of Performance Relative Advantage of suboptimal demonstrations to specify the ranking order. Second, for model training, we reconstruct the objective function of GAIL and use an experience replay buffer, enabling the agent to learn implicit features and ranking information from the ranked suboptimal demonstration set and possess the ability to outperform the demonstrator. Experiments show that in Mujoco’s tasks, our method can learn from a suboptimal demonstration set and can achieve better performance than baseline methods.
标题: Explainable Hierarchical Imitation Learning for Robotic Drink Pouring
作者: Dandan Zhang, Qiang Li, Yu Zheng
PubTime: 2022-10
Downlink: https://ieeexplore.ieee.org/document/9667114/
Journal: IEEE Transactions on Automation Science and Engineering
中文摘要: 准确地将饮料倒入各种容器是服务机器人的一项必备技能。然而，倒酒是一个动态过程，很难建模。用于实现自主机器人浇注的传统深度模仿学习技术具有固有的黑盒效应，并且需要大量的演示数据用于模型训练。为了解决这些问题，本文提出了一种可解释的分层模仿学习（EHIL）方法，使机器人可以学习高级一般知识，并在多个倒酒场景中执行低级动作。此外，使用EHIL方法，可以为任务执行构建逻辑图，通过该逻辑图，可以向用户解释动作生成的决策过程，并且可以追踪失败的原因。基于逻辑图，该框架可被操纵以实现不同的目标，同时对看不见的场景的适应性可以以可解释的方式实现。通过一系列实验验证了该方法的有效性。结果表明，EHIL在成功率、适应性、可操作性和可解释性方面优于传统的行为克隆方法。从业者注意——倒液体是人们日常生活和所有湿实验室行业中的常见活动。饮料倾倒动态控制很难建模，而流量的准确感知具有挑战性。为了使机器人能够通过观察人类演示在未知动态下进行学习，可以使用深度模仿学习。针对传统深度神经网络的局限性，提出了一种可解释的分层模仿学习（EHIL）方法。所提出的方法使机器人能够学习一系列合理的浇注阶段来执行任务，而不是简单地通过传统的行为克隆来执行任务。这样，可以保证可解释性和安全性。可操作性可以通过重构逻辑图来实现。本研究的目标是通过学习方法获得饮料的倾倒动力学，实现饮料从源容器到各种目标容器的精确快速倾倒，具有可靠的性能、适应性、可操作性和可解释性。
摘要: To accurately pour drinks into various containers is an essential skill for service robots. However, drink pouring is a dynamic process and difficult to model. Traditional deep imitation learning techniques for implementing autonomous robotic pouring have an inherent black-box effect and require a large amount of demonstration data for model training. To address these issues, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper such that a robot can learn high-level general knowledge and execute low-level actions across multiple drink pouring scenarios. Moreover, with the EHIL method, a logical graph can be constructed for task execution, through which the decision-making process for action generation can be made explainable to users and the causes of failure can be traced out. Based on the logical graph, the framework is manipulable to achieve different targets while the adaptability to unseen scenarios can be achieved in an explainable manner. A series of experiments have been conducted to verify the effectiveness of the proposed method. Results indicate that EHIL outperforms the traditional behavior cloning method in terms of success rate, adaptability, manipulability, and explainability. Note to Practitioners—Pouring liquids is a common activity in people’s daily lives and all wet-lab industries. Drink pouring dynamic control is difficult to model, while the accurate perception of flow is challenging. To enable the robot to learn under unknown dynamics via observing the human demonstration, deep imitation learning can be used. To address the limitations of traditional deep neural networks, an Explainable Hierarchical Imitation Learning (EHIL) method is proposed in this paper. The proposed method enables the robot to learn a sequence of reasonable pouring phases for performing the task rather than simply execute the task via traditional behavior cloning. In this way, explainability and safety can be ensured. Manipulability can be achieved by reconstructing the logical graph. The target of this research is to obtain pouring dynamics via the learning method and realize the precise and quick pouring of drink from the source containers to various targeted containers with reliable performance, adaptability, manipulability, and explainability.
== Embodied Artificial Intelligence@robotic agent@human robot interaction ==
标题: Would You Go to a Virtual Doctor? A Systematic Literature Review on User Preferences for Embodied Virtual Agents in Healthcare
作者: Lucie Kruse, Julia Hertel, Fariba Mostajeran
PubTime: 2023-10
Downlink: https://ieeexplore.ieee.org/document/10316479/
Journal: 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
中文摘要: 医疗虚拟代理（VAs）在支持患者实现其健康目标方面具有巨大的潜力，尤其是在生理和心理治疗需求超过医疗服务能力的时候或地区。为了创建一个可接受的现场诊断、治疗和咨询的补充，理解代理人的视觉表现、行为和责任等因素对创建值得信赖的人——代理人关系的影响是至关重要的。为了深入了解这些因素，我们进行了系统的文献综述，包括59篇关于医学领域具体化增值服务的论文。我们的综述集中在VAs在医学中的应用领域和作用，以及用于显示它们的技术。使用主题分析，我们讨论了我们在用户偏好方面的发现，以及在与医疗增值服务的互动中面临的潜力和障碍。关于视觉表现，指出了用户在外观和交流方式方面的定制愿望。同样重要的是，代理的信息建立在可靠的来源上，它们具有激励性并适应用户的知识。最后，我们的结果确定了研究差距，特别是关于人工智能的技术实现和使用。
摘要: Medical virtual agents (VAs) hold great potential to support patients in achieving their health goals, especially at times or in regions where the demand for physiological and psychological therapy exceeds the capacity of medical services. To create an accepted complement to on-site diagnosis, treatment, and counseling, it is critical to understand the impact of factors such as the agent’s visual representation, behavior, and responsibilities on creating a trustworthy human-agent relationship. To gain insights into these factors, we conducted a systematic literature review including 59 papers on embodied VAs in the medical domain. Our review focused on the application fields and the role of VAs in medicine, as well as the technology used to display them. Using thematic analysis, we discuss our findings in terms of user preferences, as well as potentials and barriers faced in the interaction with medical VAs. Concerning the visual representation, the users’ wish for customization in terms of appearance and communication modalities was pointed out. It was also important that the agent’s information builds up on trustworthy sources, that they are motivating and adapted to the users’ knowledge. Finally, our results identify research gaps, in particular regarding the technological implementation and the use of artificial intelligence.
标题: Human-in-the-Loop Embodied Intelligence With Interactive Simulation Environment for Surgical Robot Learning
作者: Yonghao Long, Wang Wei, Tao Huang
PubTime: 2023-08
Downlink: https://ieeexplore.ieee.org/document/10146515/
Journal: IEEE Robotics and Automation Letters
中文摘要: 手术机器人自动化在过去十年中吸引了越来越多的研究兴趣，预计其潜力将使外科医生、护士和患者受益。最近，具身智能的学习范式已经证明了为各种复杂任务学习良好控制策略的有前途的能力，其中具身人工智能模拟器在促进相关研究方面发挥着重要作用。然而，现有的手术机器人开源模拟器仍然不足以支持人类通过物理输入设备进行交互，这进一步限制了对人类演示如何影响政策学习的有效研究。在这项工作中，我们研究了人在回路体现智能与一个新的交互式仿真平台的手术机器人学习。具体来说，我们基于之前发布的SurRoL模拟器建立了我们的平台，该平台具有几个共同开发的新功能，允许通过输入设备进行高质量的人类交互。我们用设计的新特征展示了我们的模拟环境的改进，并通过使用人类演示和强化学习作为代表性的例子来验证将人的因素纳入具身智能的有效性。在学习效率方面获得了有希望的结果。最后，我们开发并发布了五个新的手术机器人训练任务，希望通过这些任务为未来手术具体化智能的研究铺平道路。
摘要: Surgical robot automation has attracted increasing research interest over the past decade, expecting its potential to benefit surgeons, nurses and patients. Recently, the learning paradigm of embodied intelligence has demonstrated promising ability to learn good control policies for various complex tasks, where embodied AI simulators play an essential role to facilitate relevant research. However, existing open-sourced simulators for surgical robot are still not sufficiently supporting human interactions through physical input devices, which further limits effective investigations on how the human demonstrations would affect policy learning. In this work, we study human-in-the-loop embodied intelligence with a new interactive simulation platform for surgical robot learning. Specifically, we establish our platform based on our previously released SurRoL simulator with several new features co-developed to allow high-quality human interaction via an input device. We showcase the improvement of our simulation environment with the designed new features, and validate effectiveness of incorporating human factors in embodied intelligence through the use of human demonstrations and reinforcement learning as a representative example. Promising results are obtained in terms of learning efficiency. Lastly, five new surgical robot training tasks are developed and released, with which we hope to pave the way for future research on surgical embodied intelligence.
标题: Tempering Transparency in Human-Robot Interaction
作者: Kantwon Rogers, Ayanna Howard
PubTime: 2023-05
Downlink: https://ieeexplore.ieee.org/document/10154942/
Journal: 2023 IEEE International Symposium on Ethics in Engineering, Science, and Technology (ETHICS)
中文摘要: 近年来，研究人员和政府对人工智能（AI）系统中透明度和可解释性的检查和监管方面特别感兴趣。如果人类能够理解其行为背后的机制，并利用这种理解来预测未来的行为，那么人工智能系统就是“透明的”，而可解释的人工智能的目标是以人类能够理解的方式阐明人工智能系统的行为。随着这种兴趣的增加，研究对算法透明性的好处和解释提出了相互矛盾的观点[1]。此外，研究还强调了算法透明度的政策实现中的缺陷，这些缺陷通常过于模糊，并经常导致采用不足[2]。即使有这些透明度的陷阱，似乎许多社会的默认观点是人工智能系统应该变得更加透明和可解释；然而，我们认为这一立场还需要更多的怀疑。特别是，我们认为，作为一种反叙事，考虑探索计算中一个需要缺乏透明度的新兴领域——欺骗性人工智能——是一种有益的练习。新发展的研究领域与创造（有意或无意）学习欺骗人类和其他人工智能代理的人工智能代理有关。在这里，我们将欺骗定义为“选择行动来操纵信念以利用错误推论的过程”[3]，我们将其与“撒谎”互换使用。虽然在具身代理中可能存在欺骗的物理设计方面，例如机器人的拟人化和动物化[4]，[5]，但这里我们希望专注于与人工智能代理的话语和动作相关的欺骗。从表面上看，欺骗性人工智能代理的想法似乎并不容易有益；然而，还有额外的努力来创造能够融入我们社会的人工智能代理。鉴于欺骗是许多人类和动物群体的基础部分，一些人认为，赋予人工智能代理学习欺骗的能力对于他们真正有效地互动是必要和不可避免的[6]，[7]。事实上，人们已经发现，当在人类数据上训练系统时，欺骗可能是一种突发行为[8]——从而加强了欺骗行为是与人类互动的一部分的概念。此外，先前的研究表明，人工智能欺骗，而不是透明的真实性，可以在人机交互中带来更好的结果[9]-[11]。然而，欺骗当然有明显的缺点，包括侵蚀信任[12]-[15]和降低期望的再利用[12]，[15]。由于这些负面影响，以及恶意使用的可能性，一些人建议需要完全真实的代理[16]。然而，由于人类——人工智能代理交互中欺骗的影响（短期和长期）还处于起步阶段，并且缺乏知识，目前不可能明确确定鼓励或禁止这种做法的持久影响。鉴于透明度和可解释性与欺骗是有争议的，尽管这两种想法都不是完全有益或有害的，但这在确定人工智能代理应该如何行为的伦理和监管考虑时提出了重要的细微差别[17]。因此，这项工作的目标是将人工智能欺骗作为一种反叙事来平衡透明度和可解释性与其他考虑因素，以激发关于如何在设计与人类互动的人工智能系统时，对我们选择的不可预见的后果采取主动而不是被动的讨论。
摘要: In recent years, particular interest has been taken by researchers and governments in examining and regulating aspects of transparency and explainability within artificially intelligent (AI) system. An AI system is “transparent” if humans can understand the mechanisms behind its behavior and use this understanding to make predictions about future behavior while the goal of explainable AI is to clarify an AI system’s actions in a way that humans can understand. With this increased interest, research has presented conflicting views on the benefits of algorithmic transparency and explanations [1]. Moreover, research has also highlighted flaws within policy implementations of algorithmic transparency which generally remain too vague and often results in deficient adoption [2]. Even with these pitfalls of transparency, it seems as if the default view of many societies is that AI systems should be made more transparent and explainable; however, we argue that there needs to exist added skepticism of this position. In particular, we believe it is a useful exercise to consider exploring, as a counternarrative, an emerging area within computing that necessitates a lack of transparency-deceptive AI. The newly evolving area of research pertains to the creation (intentionally or not) of AI agents that learn to deceive humans and other AI agents. Here we define deception as “the process by which actions are chosen to manipulate beliefs so as to take advantage of the erroneous inferences” [3] and we use this interchangeably with “lying”. While there may be physically designed aspects of deception in embodied agents, such as the anthropomorphism and zoomorphism of robots [4], [5], here we wish to focus on deception related to utterances and actions of AI agents. On its surface, the idea of deceptive AI agents may not readily seem beneficial; however, there exists added effort to create AI agents that are able to be integrated socially within our societies. Seeing as deception is a foundational part of many human and animal groups, some argue that giving AI agents the ability to learn to deceive is necessary and inevitable for them to truly interact effectively [6], [7]. In fact, it has been found that deception can be an emergent behavior when training systems on human data [8]-thus strengthening the notion that behaving deceptively is a part of what it means to interact with humans. Moreover, prior research has shown that AI deception, rather than transparent truthfulness, can lead to better outcomes in human-robot interactions [9]–[11]. However, deception does of course have obvious drawbacks including an erosion of trust [12]–[15] and decreasing desired reutilization [12], [15]. Because of these negative aspects, and the clear possibly of malicious usage, some suggest the need for entirely truthful agents [16]. However, due to the infancy and lack of knowledge of the effects (short and long term) of deception within human-AI agent interaction, it is currently not possible to definitively determine the lasting implications of either encouraging or banning the practice. Given that transparency and explainability are in contention with deception, while also neither of the ideas are entirely beneficial nor detrimental, this presents important nuance when determining ethical and regulatory considerations of how AI agents should behave [17]. As such, the goal of this work is to present AI deception as a counternarrative to balance transparency and explainability with other considerations to spur discussions on how to be proactive, rather than reactive, to unforeseen consequences of our choices when designing AI systems that interact with humans.
标题: On the Plane: A Roleplaying Game for Simulating Ingroup-Outgroup Biases in Virtual Reality
作者: Caglar Yildirim, D. Fox Harrell
PubTime: 2022-12
Downlink: https://ieeexplore.ieee.org/document/10024468/
Journal: 2022 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)
摘要: Many conflicts emanate from failure to understand others’ perspectives. Computationally supported roleplaying games have the potential to promote successful perspective taking. When implemented with the affordances of virtual reality, nuances of embodied communication in roleplaying can be more robustly modeled. In this paper, we describe the design and development of a roleplaying game aimed at simulating ingroup-outgroup biases with the goal of supporting positive perspective taking in virtual reality: On the Plane. The game presents players with a simulation of air travel experience, from airport security screening to in-flight events. On the Plane affords the ability to experience the simulation as different characters, supporting both ingroup and outgroup perspectives. We describe how the game is structured to simulate and challenge ingroupoutgroup biases within the context of xenophobia and lay out our plans for future research using On the Plane to promote positive perspective transformation (e.g., challenging one’s own ill-founded preconceptions).
标题: Multimodal Embodied Conversational Agents: A discussion of architectures, frameworks and modules for commercial applications
作者: Kumar Shubham, Laxmi Narayen Nagarajan Venkatesan, Dinesh Babu Jayagopi
PubTime: 2022-12
Downlink: https://ieeexplore.ieee.org/document/10024482/
Journal: 2022 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR)
摘要: With the recent advancements in automated communication technology, many traditional businesses that rely on face-to-face communication have shifted to online portals. However, these online platforms often lack the personal touch essential for customer service. Research has shown that face-to- face communication is essential for building trust and empathy with customers. A multimodal embodied conversation agent (ECA) can fill this void in commercial applications. Such a platform provides tools to understand the user’s mental state by analyzing their verbal and non-verbal behaviour and allows a human-like avatar to take necessary action based on the context of the conversation and as per social norms. However, the literature to understand the impact of ECA agents on commercial applications is limited because of the issues related to platform and scalability. In our work, we discuss some existing work that tries to solve the issues related to scalability and infrastructure. We also provide an overview of the components required for developing ECAs and their deployment in various applications.
标题: Editorial Introduction to the Special Issue on Embodied Intelligence
作者: Fumiya Iida, Josie Hughes
PubTime: 2022-08
Downlink: https://ieeexplore.ieee.org/document/10301933/
Journal: Artificial Life
摘要:
== Object Detection@ Segmentation@Open vocabulary detection@SAM ==
标题: Automated Wheat Disease Detection using Deep Learning: an Object Detection and Classification Approach
作者: Sepideh Etaati, Javad Khoramdel, Esmaeil Najafi
PubTime: 2023-12
Downlink: https://ieeexplore.ieee.org/document/10412538/
Journal: 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM)
中文摘要: 小麦作物是农业中至关重要的主食，但其产量经常受到各种疾病的影响。随着全球人口的增长，适应性农业措施至关重要，小麦病害的早期检测变得至关重要。利用深度学习技术可以为疾病检测和分类提供实用的解决方案。本文探讨了自动检测小麦穗部病害的不同方法，并确定了最佳方法。使用了两个不同的数据集：用于对象检测的全球麦穗检测（GWHD）数据集和用于分类的大型小麦疾病分类数据集（LWDC）。YOLOv4目标检测网络在GWHD上训练，实现了91%的平均精度（mAP）。训练后的权重用于域转移，以便在LWDC数据集上进行训练，用于进一步探索，其中使用三个类和COCO的预训练权重进行训练会产生更好的地图结果。此外，在LWDC小麦病害分类数据集上评估了五个CNN模型，包括VGG19、ResNet50、EfficientNet-BO、NASNetMobile和NASNetLarge。VGG19成为表现最佳的模型，准确地对各种小麦病害进行分类，平均F1得分为95%。用于对象检测的YOLOv4和用于分类的VGG19的组合提供了有希望的结果，为小麦作物的精准农业和早期疾病检测提供了有价值的见解。
摘要: The wheat crop is a crucial staple in agriculture, but its yield is often compromised by various diseases. With a growing global population, adaptive agricultural practices are essential, and early detection of wheat diseases becomes vital. Utilizing deep learning techniques can offer practical solutions for disease detection and classification. This paper explores different approaches to automate wheat head disease detection and identifies the best methods. Two distinct datasets are used: the Global Wheat Head Detection (GWHD) dataset for object detection and the Large Wheat Disease Classification Dataset (LWDC) for classification. The YOLOv4 object detection network is trained on GWHD, achieving a mean Average Precision (mAP) of 91%. The trained weights are used for domain transfer for training on LWDC dataset for further exploration, where training with three classes and COCO’s pre-trained weights yields superior mAP results. Additionally, five CNN models, including VGG19, ResNet50, EfficientNet-BO, NASNetMobile, and NASNetLarge, are evaluated on LWDC dataset for wheat disease classification. VGG19 emerges as the top-performing model, accurately classifying various wheat diseases, achieving an average F1 score of 95%. The combination of YOLOv4 for object detection and VGG19 for classification presents promising results, offering valuable insights for precision agriculture and early disease detection in wheat crops.
标题: The Application of Machine Vision Algorithm in Obstacle Recognition and Navigation of Logistics Robots
作者: Lixun Liu, Rongcheng Zhou
PubTime: 2023-12
Downlink: https://ieeexplore.ieee.org/document/10435838/
Journal: 2023 3rd International Conference on Mobile Networks and Wireless Communications (ICMNWC)
中文摘要: 在医学成像、对象检测和视频监控等各个领域，多视图自然语言查询系统利用图像数据提供更全面的视角，允许用户直观地查询和获取信息。由于硬编码匹配规则方法缺乏对自然语言的深入理解，查询结果与用户意图不符，难以满足实际应用需求。因此，本文介绍了机器视觉算法的优化和改进。通过两组仿真实验，可以得出以下结论：与传统方法相比，障碍精度指标综合平均提高11.5%左右，导航效率指标综合平均提高13.35%。该研究为物流行业注入了创新动力，使物流机器人能够在复杂环境中快速准确地感知和导航障碍物，从而提高其在仓储和分拣等任务中的智能水平，为提高物流效率和降低成本做出了实质性贡献。
摘要: In various fields such as medical imaging, object detection, and video surveillance, multi view natural language query systems utilize image data to provide a more comprehensive perspective, allowing users to intuitively query and obtain information. Due to the lack of a deep understanding of natural language in the hard coded matching rule method, the query results do not match the user’s intentions and are difficult to meet practical application needs. Therefore, this article introduces machine vision algorithms for optimization and improvement. Through two sets of simulation experiments, the following conclusion can be drawn: compared with traditional methods, the comprehensive average improvement of obstacle accuracy index is about 11.5%, while the comprehensive average improvement of navigation efficiency index is 13.35%. This study injects innovative momentum into the logistics industry, enabling logistics robots to quickly and accurately perceive and navigate obstacles in complex environments, thereby enhancing their intelligence level in tasks such as warehousing and sorting, and making substantial contributions to improving logistics efficiency and reducing costs.
标题: Real-Time Object Detection and Audio Feedback for the Visually Impaired
作者: Ayan Ravindra Jambhulkar, Akshay Rameshbhai Gajera, Chirag Manoj Bhavsar
PubTime: 2023-08
Downlink: https://ieeexplore.ieee.org/document/10269899/
Journal: 2023 3rd Asian Conference on Innovation in Technology (ASIANCON)
中文摘要: 视障人士在日常生活中面临许多挑战，包括独立识别和导航周围环境的能力。基于计算机视觉的对象检测技术已经显示出通过实时检测和分类对象来帮助视力受损者的结果。在本文中，我们使用了一个实时物体检测和音频反馈系统，该系统为视障人士提供音频反馈，以便在他们的周围环境中识别和导航。所提出的系统使用YOLO_v3算法和MS COCO数据集来实时检测和分类对象，并提供相应的音频反馈。我们使用gTTS（Google文本到语音）API来生成音频反馈。音频反馈是使用音频处理技术和深度学习算法生成的。我们对数据集进行了评估，平均检测准确率达到90%。所提出的系统为增强视障个人的可访问性和独立性提供了实用而有效的解决方案，并展示了将先进的深度学习算法和数据集用于实时对象检测和音频反馈系统的潜力。
摘要: Visually impaired individuals face numerous challenges in their daily lives, including the ability to identify and navigate through their surroundings independently. Object detection techniques based on computer vision have shown results in helping the visually impaired by detecting and classifying objects in real-time. In this paper, we used a realtime object detection and audio feedback system that provides audio feedback to the visually impaired for identifying and navigating in their surroundings. The proposed system uses the YOLO_v3 algorithm with the MS COCO dataset to detect and classify objects in real-time and provide corresponding audio feedback. We used gTTS (Google Text to Speech) API for generating the audio feedback. The audio feedback is generated using an audio processing techniques and deep learning algorithms. We evaluated on a dataset, and achieved an average detection accuracy of 90%. The proposed system provides a practical and effective solution for enhancing accessibility and independence for visually impaired individuals, and demonstrates the potential of using advanced deep learning algorithms and datasets for real-time object detection and audio feedback systems.
标题: UAV-YOLOX: An Accurate Object Detection Method for UAV Images
作者: Qi Wang, Fan Wang, Wen Zhang
PubTime: 2023-04
Downlink: https://ieeexplore.ieee.org/document/10150775/
Journal: 2023 8th International Conference on Computer and Communication Systems (ICCCS)
中文摘要: 无人机（UAV）图像的目标检测已经成为计算机视觉领域的热点。精确的目标检测对于丰富的无人机用途具有重要意义。然而，UVA图像中的微小物体和复杂背景对物体检测的性能提出了挑战。为了应对这些挑战，在本文中，我们提出了一种用于无人机图像的精确目标检测方法，称为UAV-YOLOX，通过提出一些改进来修改最先进的YOLOX检测器。首先，采用Aug-Rotation数据增强方法，通过缓解无人机不同拍摄角度导致数据集中相似物体角度不一致的问题，增强卷积神经网络的旋转不变性。然后，设计了加权循环特征金字塔，避免了传统特征金字塔结构在融合特征时丢失高级语义特征的问题，从而提高了目标检测的性能。最后，在预测头部分，本文提出了一种注意头结构，将边界框的回归和分类作为两个任务。基于注意机制的预测头可以针对不同的任务学习不同的特征。在公共数据集和收集数据集上的实验结果表明，该方法获得了更高的精度
摘要: Object detection for Unmanned Aerial Vehicle (UAV) images has become a hotspot in the field of computer vision. Accurate object detection is significant for abundant UAV utilities. However, tiny objects and complex backgrounds in UVA images challenge the performance of object detection. To address the challenges, in this paper, we propose an accurate object detection method for UAV images, named UAV-YOLOX, by proposing some improvements to modify the state-of-the-art YOLOX detector. First, an Aug-Rotation data enhancement method is used to enhance the rotation invariance of the convolutional neural network, by alleviating inconsistent angles of similar objects in the dataset caused by different shooting angles of UAV. Then, a weighted cyclic feature pyramid is designed to improve the performance of object detection, by avoiding the loss of high-level semantic features when fusing features in a traditional feature pyramid structure. Finally, in the part of prediction head, this paper proposes an attention head structure, which regards the regression and classification of the bounding box as two tasks. The prediction head based on the attention mechanism can learn different features for different tasks. Experimental results on the public datasets and collected datasets show that the proposed method achieves much higher accuracy.
标题: Research Advanced in Deep Learning Object Detection
作者: Mingzhe Zhou
PubTime: 2022-12
Downlink: https://ieeexplore.ieee.org/document/10016018/
Journal: 2022 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS)
摘要: Object detection aims to locate and assign various predefined substances from images to corresponding classifications. Benefit from the rapid evolution of deep learning, target detection algorithms based on convolutional neural networks have achieved breakthroughs in both accuracy and efficiency. Based on the detailed literature research and analysis, in this paper, we make an integrated assess of the research progress of object detection. Specifically, we first introduce the existing representative algorithms from two-stage detection framework to one-stage detection framework. Then a battery of experiments are conducted to analyze the implementation of different detection algorithms on some common datasets. Finally, we summarize the main challenges and give an outlook on future research development for object detection.
标题: A Multi-object Detection Sampling Algorithm For Large Scenes
作者: Liang Jin, Xiaochuan Li, Baoyu Fan
PubTime: 2022-11
Downlink: https://ieeexplore.ieee.org/document/10010614/
Journal: 2022 IEEE 13th International Symposium on Parallel Architectures, Algorithms and Programming (PAAP)
摘要: Multi-object detection in large scenes aims to find objects in images, which usually contain more than one billion pixels. Based on the concept of dividing and conquering, the state-of-the-art (SOTA) methods slice the super-resolution image into patches first and then lower the image solution to detect objects later. The advantage of this method is that it can adapt quickly to regular detection algorithms. However, a set of parameters needs to be set manually, such as the size of sliding windows and overlap, which is quite hard to fit all scenarios. It may result in a loss of samples located at the boundary of the sliding window and the oversampling of inefficient samples that appear within the overlap. In this paper, we propose a object-oriented image sampling algorithm based on anchor boxes during training and multi-scale pyramids during inference. Inspired by the mature object detection baseline Scale-YOLOv4, we present more tricks to fit large scenes. The accuracy can reach 66%, which is 24 points higher than the CascadeRCNN model of the official backbone network ResNet50. Finally, we have won first place in the PANDA object detection tracking using this method.
专属领域论文订阅
关注{晓理紫|小李子}，每日更新论文，如感兴趣，请转发给有需要的同学，谢谢支持
如果你感觉对你有所帮助，请关注我，每日准时为你推送最新论文。
