Context information (CI)-aware beam management solutions have provided promising performance. We propose a location- and orientation-aware beam selection framework, which uses machine learning (ML) power for leveraging the CI. We propose several deep neural network (DNN) architectures for the ML model, which are suitable for different amounts of training samples due to including different numbers of trainable parameters. Evaluations with hand-held multi-panel devices reveal the usefulness of the terminal location and orientation for ML-enabled beam and panel selection, which provides certainty in the performance offered by the ML for more realistic configurations. In another study, the self-blockage impact on the context-aware beam selection approach is evaluated. Due to the strong relation between hand blockage effects and terminal orientation, context-aware methods can leverage the orientation information to recommend beams with the lowest possibility of blockage. Furthermore, we use the transfer learning technique to reduce the concern about the generalization and scalability aspects of context-aware ML-based solutions. In addition, this thesis proposes a novel device-agnostic beam selection framework that enables the use of a generic ML model for different device codebooks/antenna configurations.