bevy_picking/lib.rs
1//! This crate provides 'picking' capabilities for the Bevy game engine, allowing pointers to
2//! interact with entities using hover, click, and drag events.
3//!
4//! ## Overview
5//!
6//! In the simplest case, this plugin allows you to click on things in the scene. However, it also
7//! allows you to express more complex interactions, like detecting when a touch input drags a UI
8//! element and drops it on a 3d mesh rendered to a different camera.
9//!
10//! Pointer events bubble up the entity hierarchy and can be used with observers, allowing you to
11//! succinctly express rich interaction behaviors by attaching pointer callbacks to entities:
12//!
13//! ```rust
14//! # use bevy_ecs::prelude::*;
15//! # use bevy_picking::prelude::*;
16//! # #[derive(Component)]
17//! # struct MyComponent;
18//! # let mut world = World::new();
19//! world.spawn(MyComponent)
20//! .observe(|mut event: On<Pointer<Click>>| {
21//! // Read the underlying pointer event data
22//! println!("Pointer {:?} was just clicked!", event.pointer_id);
23//! // Stop the event from bubbling up the entity hierarchy
24//! event.propagate(false);
25//! });
26//! ```
27//!
28//! At its core, this crate provides a robust abstraction for computing picking state regardless of
29//! pointing devices, or what you are hit testing against. It is designed to work with any input,
30//! including mouse, touch, pens, or virtual pointers controlled by gamepads.
31//!
32//! ## Expressive Events
33//!
34//! Although the events in this module (see [`events`]) can be listened to with normal
35//! `MessageReader`s, using observers is often more expressive, with less boilerplate. This is because
36//! observers allow you to attach event handling logic to specific entities, as well as make use of
37//! event bubbling.
38//!
39//! When events are generated, they bubble up the entity hierarchy starting from their target, until
40//! they reach the root or bubbling is halted with a call to
41//! [`On::propagate`](bevy_ecs::observer::On::propagate). See [`Observer`] for details.
42//!
43//! This allows you to run callbacks when any children of an entity are interacted with, and leads
44//! to succinct, expressive code:
45//!
46//! ```
47//! # use bevy_ecs::prelude::*;
48//! # use bevy_transform::prelude::*;
49//! # use bevy_picking::prelude::*;
50//! # #[derive(Message)]
51//! # struct Greeting;
52//! fn setup(mut commands: Commands) {
53//! commands.spawn(Transform::default())
54//! // Spawn your entity here, e.g. a `Mesh3d`.
55//! // When dragged, mutate the `Transform` component on the dragged target entity:
56//! .observe(|drag: On<Pointer<Drag>>, mut transforms: Query<&mut Transform>| {
57//! let mut transform = transforms.get_mut(drag.entity).unwrap();
58//! transform.rotate_local_y(drag.delta.x / 50.0);
59//! })
60//! .observe(|click: On<Pointer<Click>>, mut commands: Commands| {
61//! println!("Entity {} goes BOOM!", click.entity);
62//! commands.entity(click.entity).despawn();
63//! })
64//! .observe(|over: On<Pointer<Over>>, mut greetings: MessageWriter<Greeting>| {
65//! greetings.write(Greeting);
66//! });
67//! }
68//! ```
69//!
70//! ## Modularity
71//!
72//! #### Mix and Match Hit Testing Backends
73//!
74//! The plugin attempts to handle all the hard parts for you, all you need to do is tell it when a
75//! pointer is hitting any entities. Multiple backends can be used at the same time! [Use this
76//! simple API to write your own backend](crate::backend) in about 100 lines of code.
77//!
78//! #### Input Agnostic
79//!
80//! Picking provides a generic Pointer abstraction, which is useful for reacting to many different
81//! types of input devices. Pointers can be controlled with anything, whether it's the included
82//! mouse or touch inputs, or a custom gamepad input system you write yourself to control a virtual
83//! pointer.
84//!
85//! ## Robustness
86//!
87//! In addition to these features, this plugin also correctly handles multitouch, multiple windows,
88//! multiple cameras, viewports, and render layers. Using this as a library allows you to write a
89//! picking backend that can interoperate with any other picking backend.
90//!
91//! # Getting Started
92//!
93//! TODO: This section will need to be re-written once more backends are introduced.
94//!
95//! #### Next Steps
96//!
97//! To learn more, take a look at the examples in the
98//! [examples](https://github.com/bevyengine/bevy/tree/main/examples/picking). You can read the next
99//! section to understand how the plugin works.
100//!
101//! # The Picking Pipeline
102//!
103//! This plugin is designed to be extremely modular. To do so, it works in well-defined stages that
104//! form a pipeline, where events are used to pass data between each stage.
105//!
106//! #### Pointers ([`pointer`](mod@pointer))
107//!
108//! The first stage of the pipeline is to gather inputs and update pointers. This stage is
109//! ultimately responsible for generating [`PointerInput`](pointer::PointerInput) events. The
110//! provided crate does this automatically for mouse, touch, and pen inputs. If you wanted to
111//! implement your own pointer, controlled by some other input, you can do that here. The ordering
112//! of events within the [`PointerInput`](pointer::PointerInput) stream is meaningful for events
113//! with the same [`PointerId`](pointer::PointerId), but not between different pointers.
114//!
115//! Because pointer positions and presses are driven by these events, you can use them to mock
116//! inputs for testing.
117//!
118//! After inputs are generated, they are then collected to update the current
119//! [`PointerLocation`](pointer::PointerLocation) for each pointer.
120//!
121//! #### Backend ([`backend`])
122//!
123//! A picking backend only has one job: reading [`PointerLocation`](pointer::PointerLocation)
124//! components, and producing [`PointerHits`](backend::PointerHits). You can find all documentation
125//! and types needed to implement a backend at [`backend`].
126//!
127//! You will eventually need to choose which picking backend(s) you want to use. This crate does not
128//! supply any backends, and expects you to select some from the other bevy crates or the
129//! third-party ecosystem.
130//!
131//! It's important to understand that you can mix and match backends! For example, you might have a
132//! backend for your UI, and one for the 3d scene, with each being specialized for their purpose.
133//! Bevy provides some backends out of the box, but you can even write your own. It's been made as
134//! easy as possible intentionally; the `bevy_mod_raycast` backend is 50 lines of code.
135//!
136//! #### Hover ([`hover`])
137//!
138//! The next step is to use the data from the backends, combine and sort the results, and determine
139//! what each cursor is hovering over, producing a [`HoverMap`](`crate::hover::HoverMap`). Note that
140//! just because a pointer is over an entity, it is not necessarily *hovering* that entity. Although
141//! multiple backends may be reporting that a pointer is hitting an entity, the hover system needs
142//! to determine which entities are actually being hovered by this pointer based on the pick depth,
143//! order of the backend, and the optional [`Pickable`] component of the entity. In other
144//! words, if one entity is in front of another, usually only the topmost one will be hovered.
145//!
146//! #### Events ([`events`])
147//!
148//! In the final step, the high-level pointer events are generated, such as events that trigger when
149//! a pointer hovers or clicks an entity. These simple events are then used to generate more complex
150//! events for dragging and dropping.
151//!
152//! Because it is completely agnostic to the earlier stages of the pipeline, you can easily extend
153//! the plugin with arbitrary backends and input methods, yet still use all the high level features.
154
155#![deny(missing_docs)]
156
157extern crate alloc;
158
159pub mod backend;
160pub mod events;
161pub mod hover;
162pub mod input;
163#[cfg(feature = "bevy_mesh_picking_backend")]
164pub mod mesh_picking;
165pub mod pointer;
166pub mod window;
167
168use bevy_app::{prelude::*, PluginGroupBuilder};
169use bevy_ecs::prelude::*;
170use bevy_reflect::prelude::*;
171use hover::{update_is_directly_hovered, update_is_hovered};
172
173/// The picking prelude.
174///
175/// This includes the most common types in this crate, re-exported for your convenience.
176pub mod prelude {
177 #[cfg(feature = "bevy_mesh_picking_backend")]
178 #[doc(hidden)]
179 pub use crate::mesh_picking::{
180 ray_cast::{MeshRayCast, MeshRayCastSettings, RayCastBackfaces, RayCastVisibility},
181 MeshPickingCamera, MeshPickingPlugin, MeshPickingSettings,
182 };
183 #[doc(hidden)]
184 pub use crate::{
185 events::*, input::PointerInputPlugin, pointer::PointerButton, DefaultPickingPlugins,
186 InteractionPlugin, Pickable, PickingPlugin,
187 };
188}
189
190/// An optional component that marks an entity as usable by a backend, and overrides default
191/// picking behavior for an entity.
192///
193/// This allows you to make an entity non-hoverable, or allow items below it to be hovered.
194///
195/// See the documentation on the fields for more details.
196#[derive(Component, Debug, Clone, Reflect, PartialEq, Eq)]
197#[reflect(Component, Default, Debug, PartialEq, Clone)]
198pub struct Pickable {
199 /// Should this entity block entities below it from being picked?
200 ///
201 /// This is useful if you want picking to continue hitting entities below this one. Normally,
202 /// only the topmost entity under a pointer can be hovered, but this setting allows the pointer
203 /// to hover multiple entities, from nearest to farthest, stopping as soon as it hits an entity
204 /// that blocks lower entities.
205 ///
206 /// Note that the word "lower" here refers to entities that have been reported as hit by any
207 /// picking backend, but are at a lower depth than the current one. This is different from the
208 /// concept of event bubbling, as it works irrespective of the entity hierarchy.
209 ///
210 /// For example, if a pointer is over a UI element, as well as a 3d mesh, backends will report
211 /// hits for both of these entities. Additionally, the hits will be sorted by the camera order,
212 /// so if the UI is drawing on top of the 3d mesh, the UI will be "above" the mesh. When hovering
213 /// is computed, the UI element will be checked first to see if it this field is set to block
214 /// lower entities. If it does (default), the hovering system will stop there, and only the UI
215 /// element will be marked as hovered. However, if this field is set to `false`, both the UI
216 /// element *and* the mesh will be marked as hovered.
217 ///
218 /// Entities without the [`Pickable`] component will block by default.
219 pub should_block_lower: bool,
220
221 /// If this is set to `false` and `should_block_lower` is set to true, this entity will block
222 /// lower entities from being interacted and at the same time will itself not emit any events.
223 ///
224 /// Note that the word "lower" here refers to entities that have been reported as hit by any
225 /// picking backend, but are at a lower depth than the current one. This is different from the
226 /// concept of event bubbling, as it works irrespective of the entity hierarchy.
227 ///
228 /// For example, if a pointer is over a UI element, and this field is set to `false`, it will
229 /// not be marked as hovered, and consequently will not emit events nor will any picking
230 /// components mark it as hovered. This can be combined with the other field
231 /// [`Self::should_block_lower`], which is orthogonal to this one.
232 ///
233 /// Entities without the [`Pickable`] component are hoverable by default.
234 pub is_hoverable: bool,
235}
236
237impl Pickable {
238 /// This entity will not block entities beneath it, nor will it emit events.
239 ///
240 /// If a backend reports this entity as being hit, the picking plugin will completely ignore it.
241 pub const IGNORE: Self = Self {
242 should_block_lower: false,
243 is_hoverable: false,
244 };
245}
246
247impl Default for Pickable {
248 fn default() -> Self {
249 Self {
250 should_block_lower: true,
251 is_hoverable: true,
252 }
253 }
254}
255
256/// Groups the stages of the picking process under shared labels.
257#[derive(Debug, Hash, PartialEq, Eq, Clone, SystemSet)]
258pub enum PickingSystems {
259 /// Produces pointer input events. In the [`First`] schedule.
260 Input,
261 /// Runs after input events are generated but before commands are flushed. In the [`First`]
262 /// schedule.
263 PostInput,
264 /// Receives and processes pointer input events. In the [`PreUpdate`] schedule.
265 ProcessInput,
266 /// Reads inputs and produces [`backend::PointerHits`]s. In the [`PreUpdate`] schedule.
267 Backend,
268 /// Reads [`backend::PointerHits`]s, and updates the hovermap, selection, and highlighting states. In
269 /// the [`PreUpdate`] schedule.
270 Hover,
271 /// Runs after all the [`PickingSystems::Hover`] systems are done, before event listeners are triggered. In the
272 /// [`PreUpdate`] schedule.
273 PostHover,
274 /// Runs after all other picking sets. In the [`PreUpdate`] schedule.
275 Last,
276}
277
278/// Deprecated alias for [`PickingSystems`].
279#[deprecated(since = "0.17.0", note = "Renamed to `PickingSystems`.")]
280pub type PickSet = PickingSystems;
281
282/// One plugin that contains the [`PointerInputPlugin`](input::PointerInputPlugin), [`PickingPlugin`]
283/// and the [`InteractionPlugin`], this is probably the plugin that will be most used.
284///
285/// Note: for any of these plugins to work, they require a picking backend to be active,
286/// The picking backend is responsible to turn an input, into a [`PointerHits`](`crate::backend::PointerHits`)
287/// that [`PickingPlugin`] and [`InteractionPlugin`] will refine into [`bevy_ecs::observer::On`]s.
288#[derive(Default)]
289pub struct DefaultPickingPlugins;
290
291impl PluginGroup for DefaultPickingPlugins {
292 fn build(self) -> PluginGroupBuilder {
293 PluginGroupBuilder::start::<Self>()
294 .add(input::PointerInputPlugin)
295 .add(PickingPlugin)
296 .add(InteractionPlugin)
297 }
298}
299
300#[derive(Copy, Clone, Debug, Resource, Reflect)]
301#[reflect(Resource, Default, Debug, Clone)]
302/// Controls the behavior of picking
303///
304/// ## Custom initialization
305/// ```
306/// # use bevy_app::App;
307/// # use bevy_picking::{PickingSettings, PickingPlugin};
308/// App::new()
309/// .insert_resource(PickingSettings {
310/// is_enabled: true,
311/// is_input_enabled: false,
312/// is_hover_enabled: true,
313/// is_window_picking_enabled: false,
314/// })
315/// // or DefaultPlugins
316/// .add_plugins(PickingPlugin);
317/// ```
318pub struct PickingSettings {
319 /// Enables and disables all picking features.
320 pub is_enabled: bool,
321 /// Enables and disables input collection.
322 pub is_input_enabled: bool,
323 /// Enables and disables updating interaction states of entities.
324 pub is_hover_enabled: bool,
325 /// Enables or disables picking for window entities.
326 pub is_window_picking_enabled: bool,
327}
328
329impl PickingSettings {
330 /// Whether or not input collection systems should be running.
331 pub fn input_should_run(state: Res<Self>) -> bool {
332 state.is_input_enabled && state.is_enabled
333 }
334
335 /// Whether or not systems updating entities' [`PickingInteraction`](hover::PickingInteraction)
336 /// component should be running.
337 pub fn hover_should_run(state: Res<Self>) -> bool {
338 state.is_hover_enabled && state.is_enabled
339 }
340
341 /// Whether or not window entities should receive pick events.
342 pub fn window_picking_should_run(state: Res<Self>) -> bool {
343 state.is_window_picking_enabled && state.is_enabled
344 }
345}
346
347impl Default for PickingSettings {
348 fn default() -> Self {
349 Self {
350 is_enabled: true,
351 is_input_enabled: true,
352 is_hover_enabled: true,
353 is_window_picking_enabled: true,
354 }
355 }
356}
357
358/// This plugin sets up the core picking infrastructure. It receives input events, and provides the shared
359/// types used by other picking plugins.
360///
361/// Behavior of picking can be controlled by modifying [`PickingSettings`].
362///
363/// [`PickingSettings`] will be initialized with default values if it
364/// is not present at the moment this is added to the app.
365pub struct PickingPlugin;
366
367impl Plugin for PickingPlugin {
368 fn build(&self, app: &mut App) {
369 app.init_resource::<PickingSettings>()
370 .init_resource::<pointer::PointerMap>()
371 .init_resource::<backend::ray::RayMap>()
372 .add_message::<pointer::PointerInput>()
373 .add_message::<backend::PointerHits>()
374 // Rather than try to mark all current and future backends as ambiguous with each other,
375 // we allow them to send their hits in any order. These are later sorted, so submission
376 // order doesn't matter. See `PointerHits` docs for caveats.
377 .allow_ambiguous_resource::<Messages<backend::PointerHits>>()
378 .add_systems(
379 PreUpdate,
380 (
381 pointer::update_pointer_map,
382 pointer::PointerInput::receive,
383 backend::ray::RayMap::repopulate.after(pointer::PointerInput::receive),
384 )
385 .in_set(PickingSystems::ProcessInput),
386 )
387 .add_systems(
388 PreUpdate,
389 window::update_window_hits
390 .run_if(PickingSettings::window_picking_should_run)
391 .in_set(PickingSystems::Backend),
392 )
393 .configure_sets(
394 First,
395 (PickingSystems::Input, PickingSystems::PostInput)
396 .after(bevy_time::TimeSystems)
397 .after(bevy_ecs::message::MessageUpdateSystems)
398 .chain(),
399 )
400 .configure_sets(
401 PreUpdate,
402 (
403 PickingSystems::ProcessInput.run_if(PickingSettings::input_should_run),
404 PickingSystems::Backend,
405 PickingSystems::Hover.run_if(PickingSettings::hover_should_run),
406 PickingSystems::PostHover,
407 PickingSystems::Last,
408 )
409 .chain(),
410 );
411 }
412}
413
414/// Generates [`Pointer`](events::Pointer) events and handles event bubbling.
415#[derive(Default)]
416pub struct InteractionPlugin;
417
418impl Plugin for InteractionPlugin {
419 fn build(&self, app: &mut App) {
420 use events::*;
421 use hover::{generate_hovermap, update_interactions};
422
423 app.init_resource::<hover::HoverMap>()
424 .init_resource::<hover::PreviousHoverMap>()
425 .init_resource::<PointerState>()
426 .add_message::<Pointer<Cancel>>()
427 .add_message::<Pointer<Click>>()
428 .add_message::<Pointer<Press>>()
429 .add_message::<Pointer<DragDrop>>()
430 .add_message::<Pointer<DragEnd>>()
431 .add_message::<Pointer<DragEnter>>()
432 .add_message::<Pointer<Drag>>()
433 .add_message::<Pointer<DragLeave>>()
434 .add_message::<Pointer<DragOver>>()
435 .add_message::<Pointer<DragStart>>()
436 .add_message::<Pointer<Move>>()
437 .add_message::<Pointer<Out>>()
438 .add_message::<Pointer<Over>>()
439 .add_message::<Pointer<Release>>()
440 .add_message::<Pointer<Scroll>>()
441 .add_systems(
442 PreUpdate,
443 (
444 generate_hovermap,
445 update_interactions,
446 (update_is_hovered, update_is_directly_hovered),
447 pointer_events,
448 )
449 .chain()
450 .in_set(PickingSystems::Hover),
451 );
452 }
453}