Ultra-Reliable and Low Latency Communications (uRLLC) is a key enabler for many use cases of 5G and 6G including autonomous vehicles, smart grid and mobile AR/VR. We use reinforcement learning and deep learning to address allocation of resources in wireless systems such that the complexity of the environment is learnt and optimal decisions are taken by the agents. Our most recent work in the area is listed below.
[ToSG20] M. Elsayed, Melike Erol-Kantarci, B. Kantarci, L. Wu, J. Li, “Low-latency Communications for Community Resilience Microgrids: A Reinforcement Learning Approach,” in IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1091-1099, 2020.
[VTSMag19] M. Elsayed, Melike Erol-Kantarci, “AI-enabled Future Wireless Networks: Challenges, Opportunities and Open Issues,” in IEEE Vehicular Technology Magazine, Special Issue on 6G: What is Next?, Volume: 14 , Issue: 3 , pp. 70-77, Sept. 2019.
[ICC20] M. Elsayed, K. Shimotakahara, Melike Erol-Kantarci, “Machine Learning-based Inter-Beam Inter-Cell Interference Mitigation in mmWave,” accepted to IEEE ICC, Dublin, June 2020.
[Globecom19] M. Elsayed, Melike Erol-Kantarci, “Reinforcement Learning-based Joint Power and Resource Allocation for URLLC in 5G,” accepted to IEEE Globecom, Waikoloa, HI, USA, December 2019.
[SmartGridComm18] M. Elsayed, Melike Erol-Kantarci, “Deep Q-Learning for Low-Latency Tactile Applications: Microgrid Communications,” accepted to Proc of IEEE SmartGridComm Workshops, Aalborg, Denmark, October 2018.
Device-to-device (D2D) mobile communications offer a low latency solution for the exchange of information in microgrids. We have investigated the usage of reinforcement learning techniques to help D2D-enabled grid entities self-allocate their cellular resources to meet the quality of service requirements of mission-critical smart grid applications [IEEE Access 2019]. To help do this, novel simulation techniques were developed to combine the heterogeneous characteristics of power and communication system dynamics [GlobalSIP 2019].
[[GlobalSIP 2019] K.
Shimotakahara, M. Elsayed, K. Hinzer and Melike Erol-Kantarci,
"Integrated Power and Device-to-Device (D2D) Communications
Simulator for Future Power Systems," IEEE Global Conference on
Signal and Information Processing (GlobalSIP), Ottawa, ON, Canada,
[IEEE Access 2019] K. Shimotakahara, M. Elsayed, K. Hinzer and Melike Erol-Kantarci, "High-Reliability Multi-Agent Q-Learning-Based Scheduling for D2D Microgrid Communications," in IEEE Access, vol. 7, pp. 74412-74421, 2019.
Microgrids are anticipated to be a significant ingredient of the smart grid as they simplify the integration of distributed renewable energy generation units, offer reliable service, provide faster restoration capabilities and support sustainable grid operation. In [NETMAG 2011], we have proposed to organize individual microgrids in a smart microgrid network which is a dynamically built platform allowing power transfers among microgrids in order to maximize the utilization of the renewable resources and minimize dependency to the utility grid. In [GBCOMM 2011], we extended this work to include cost-aware energy transactions. The basic idea of interconnected microgrids can be used to enhance resilience of the smart grid [ElectricityJ17]. In [PIMRC19], we extended our prior studies to form microgrid coalitions to reduce power loss. We used machine learning to address the uncertainty.
[PIMRC19] M. Sadeghi, Melike Erol-Kantarci, “Power Loss Minimization in Microgrids Using Bayesian Reinforcement Learning with Coalition Formation,” in Proc. of IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Istanbul, Turkey, September 2019.
[ElectricityJ17] L. Wu, J. Li, Melike Erol Kantarci, B. Kantarci, “An Integrated Reconfigurable Control and Self-Organizing Communication Framework for Community Resilience Microgrids,” The Electricity Journal, Vol. 30, no. 4, pp. 27-34, May 2017.
[NETMAG 2011] M. Erol-Kantarci, B. Kantarci, H. T. Mouftah, `Reliable Overlay Topology Design for the Smart Microgrid Network,` accepted for publication in IEEE Network Magazine, Special issue on Communication Infrastructures for Smart Grid, September 2011.
[GBCOMM 2011] M. Erol-Kantarci, B. Kantarci, H. T. Mouftah, `Cost-Aware Smart Microgrid Network Design for a Sustainable Smart Grid,` in Proc. of IEEE GLOBECOM - Workshop on Smart Grid Communications and Networks, pp. 1223-1227, Houston, TX, USA, Dec. 5-9, 2011.
As a result of increasing popularity of augmented reality and virtual reality (AR/
VR) applications, there are significant efforts to bring AR/VR to mobile users. Parallel to
the advances in AR/VR technologies, tactile internet is gaining interest from the research
community. Both AR/VR and tactile internet applications require massive computational
capability, high communication bandwidth, and ultra-low latency that cannot be provided
with the current wireless mobile networks. By 2020, long term evolution (LTE) networks
will start to be replaced by fifth generation (5G) networks. Edge caching and mobile edge
computing are among the potential 5G technologies that bring content and computing
resources close to the users, reducing latency and load on the
[Multimedia19]S. Sukhmani, M. Sadeghi, Melike Erol-Kantarci, A. El-Saddik, “Edge Caching and Computing in 5G for Mobile AR/VR and Tactile Internet,” accepted to IEEE Multimedia, 2019.
[AdHocNets17] Melike Erol-Kantarci, S. Sukhmani, “Caching and Computing at the Edge for Mobile Augmented Reality and Virtual Reality in 5G,” ADHOCNETS, Niagara Falls, 2017.
Video upload and download are among the most popular applications for mobile users. However, those have high bandwidth demand and consume device power rapidly. Given the fact that popular contents are acquired by a large number of users at different times, it is practical to fetch those contents from the content source and store in-network. This is known as in-network caching. In the literature, latency and transport energy minimization aware in-network caching has been studied widely. Uplink power optimization which aims prolonging User Equipment (UE) lifetime have not been considered yet. To address this problem, we cache contents at wireless relays of a Heterogeneous Network (HetNet) and propose two Integer Linear Programming (ILP) models to optimize uplink power. The first ILP model, namely Minimize Uplink Power (minUP). MinUP aims to minimize uplink power when relays and their contents are known. The second ILP-based scheme is called Minimize Uplink Power and Relay Cost (minUPREC). MinUPREC jointly minimizes uplink energy and relay deployment cost. We have shown that minUPREC significantly minimizes the uplink energy incurred by UEs while minUP consumes lower caching.
[WCNC15] Melike Erol-Kantarci, "Content Caching in Small Cells with Optimized Uplink and Caching Power, " IEEE Wireless Communications and Networking Conference (WCNC), New Orleans, USA, 2015.
[ICNC15] Melike Erol-Kantarci, "Uplink Power Optimized In-Network Content Caching for HetNets, " International Conference on Computing, Networking and Communications (ICNC) - Workshop on Computing, Networking and Communications (CNC), Anaheim, CA, USA, 2015.
Plug-In Hybrid Electric Vehicles (PHEVs) have lower operating costs and lower emissions than the conventional vehicles which position them as promising candidates among the future transportation alternatives. On the other hand, they face the risk of endangering power grid stability due to their extremely high power demand. To address these challenges, in [CAMAD 2011], we have proposed an admission control scheme that runs on the distribution substation level using utility provided thresholds to provide access to the charging infrastructure. We extended this work to include differentiated admission decisions in [IEVC 2012]. In [ADJ 2014], energy transactions between delay tolerant loads and PEVs are studied. Our approach provides a convenient supply for loads via PEV batteries while addressing the mobility of vehicles and preserving fairness.
[AHJ 2014] Melike Erol-Kantarci, J. H. Sarker, H. T. Mouftah, `A Four-way-handshake Protocol for Energy Forwarding Networks in the Smart Grid`, Elsevier Ad Hoc Networks Journal, vol. 22, pp. 83-92, 2014.
[CAMAD 2011] M. Erol-Kantarci, J. Sarker, H. T. Mouftah, `Analysis of Plug-in Hybrid Electrical Vehicle Admission Control in the Smart Grid, ` International Workshop on Computer-Aided Modeling Analysis and Design of Communication Links and Networks (CAMAD), Kyoto, Japan, June 10-11, 2011.
[IEVC 2012] M. Erol-Kantarci, J. H. Sarker, H. T. Mouftah, `Quality of Service in Plug-in Electric Vehicle Charging Infrastructure, ` in Proc. of IEEE International Electric Vehicle Conference, Greenville, SC, March 4-8, 2012.
[IWCM 2011] M. Erol-Kantarci, H. T. Mouftah, `Management of PHEV Batteries in the Smart Grid: Towards a Cyber-Physical Power Infrastructure,` Workshop on Design, Modeling and Evaluation of Cyber Physical Systems (in IWCMC11), Istanbul, Turkey, July 5-8, 2011.
WSNs are immensely deployed in various environments including industrial facilities, smart homes, health systems and other cyber-physical systems. However, conventional battery-powered WSNs have significant maintenance cost as a result of replacing depleted sensor batteries. Battery-free sensor technology offers a solution to the battery replacement and disposal burden by ripping batteries off the sensors, powering sensors with stored energy in super capacitors and topping the super capacitor from ambient energy. Wireless Energy Transfer (WET) promises charging wireless sensor networks, cell phones and on-body medical devices without the need of battery replacement nor plugging in to the mains. Radio Frequency WET (RF-WET) can be used for prolonging UE lifetime in a Heterogeneous wireless network (HetNet).
[PIMRC 2014] Melike Erol-Kantarci, H. T. Mouftah, `Challenges of Wireless Power Transfer for Prolonging User Equipment (UE) Lifetime in Wireless Networks,` in Proc. of IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) Workshop on Current Challenges for Wireless Power Transfer, Washington DC, USA, 2014.
[ISCC 2014] Melike Erol-Kantarci, H. T. Mouftah, `Radio-Frequency-Based Wireless Energy Transfer in LTE-A Heterogeneous Networks, ` in Proc. of IEEE Symposium on Computers and Communications (ISCC), Portugal, June 23-26, 2014.
[ISCC 2012] M. Erol-Kantarci, H. T. Mouftah, `'Mission-Aware Placement of RF-based Power Transmitters in Wireless Sensor Networks, ` in Proc. of IEEE Symposium on Computers and Communications (ISCC), Cappadocia, Turkey, July 1-4, 2012.
For a reliable smart grid, accurate, robust monitoring and diagnosis tools are essential. Wireless Sensor Networks (WSNs) are promising candidates for monitoring the smart grid, given their capability to cover large geographic regions at low-cost. On the other hand, limited battery lifetime of the conventional WSNs create a performance bottleneck for the long-lasting smart grid monitoring tasks, especially considering that the sensor nodes may be deployed in hard to reach, harsh environments. In [TOSG 2011, VTC 2010], we have proposed a WSN-based residential demand management scheme which combines the Time Of Use (TOU) tariff from the smart meter, energy generation input from the renewable generators and provides a schedule for controllable appliances. [TOSG 2011] is listed in IEEE ComSoc Best Readings in Smart Grid Communications. This Best Readings is considered to be `a concise list of must-read books and articles` on communication networks and systems in the smart grid. In [AHNJ 2011a], we have investigated the opportunities and challenges of employing Wireless Multimedia Sensor and Actor Networks (WMSAN) in the smart grid. This paper has been listed by Elsevier Ad Hoc Networks journal as fourth most downloaded article of 2011 and ninth most cited article since publication. In [SYSJ 2013], we proposed delay-aware and fairness-aware schemes for prioritized packets, and in [TETC 2013] we extended this work to address further challenges in large-scale cyber physical systems, i.e. to enable prioritized medium access in IEEE 802.15.4 in cluster-tree topologies. In [WICOMM 2012], we have proposed the Sustainable wireless Rechargeable Sensor network (SuReSense) which employs mobile chargers that provide wireless power to multiple sensors from several landmark locations.
[SYSJ 2013] I. Al-Anbagi, Melike Erol-Kantarci, H. T. Mouftah, `Priority and Delay Aware Medium Access for Wireless Sensor Networks in the Smart Grid`, IEEE Systems Journal, vol.8, no.2, pp. 608 - 618, 2014.
[TETC 2013] I. Al-Anbagi, Melike Erol-Kantarci, H. T. Mouftah, `A Reliable IEEE 802.15.4 Model for Cyber Physical Power Grid Monitoring Systems,` to appear in IEEE Transactions on Emerging Topics in Computing, 2013.
[TOSG 2011] M. Erol-Kantarci, H. T. Mouftah, `Wireless Sensor Networks for Cost-Efficient Residential Energy Management in the Smart Grid`, IEEE Transactions on Smart Grid, vol.2, no.2, pp.314-325, June 2011.
[VTC 2010]M. Erol-Kantarci, H. T. Mouftah, `TOU-Aware Energy Management and Wireless Sensor Networks for Reducing Peak Load in Smart Grids,` Green Wireless Communications and Networks Workshop in IEEE VTC2010-Fall, Ottawa, ON, Canada, September 6-9, 2010.
[AHNJ 2011a] M. Erol-Kantarci, H. T. Mouftah, `Wireless Multimedia Sensor and Actor Networks for the Next-Generation Power Grid,` Elsevier Ad Hoc Networks Journal, vol.9 no.4, 2011, pp. 542-511.
[WICOMM 2012] M. Erol-Kantarci, H. T. Mouftah, `SuReSense: Sustainable Wireless Rechargeable Sensor Networks for the Smart Grid,` IEEE Wireless Communications, vol.19, no. 3, pp. 30-36, June 2012.
Smart grid forensic science is a newly flourishing research area that is tightly coupled with cyber and physical security of the smart grid. Post-mortem analysis of a power system after a cyber attack or after a natural disaster generally provides the most accurate comprehension of the real world vulnerabilities of the system and helps to protect the grid against similar attacks in the future as well as avoiding failures under disasters. In [COMMAG 2013], we have introduced the emerging application areas of smart grid forensic science, discussed the challenges and outlined the open issues in the topic. This paper serves as a roadmap for future smart grid forensic studies. This paper has been listed in IEEE ComSoc Technology News (CTN) in March 2013. IEEE CTN each month selects `interesting, timely, and newsworthy papers and articles from IEEE publications`.
[COMMAG 2013] M. Erol-Kantarci, H. T. Mouftah, `Smart Grid Forensic Science: Applications, Challenges and Open Issues, ` in IEEE Communications Magazine, January 2013.
Underwater research aims at providing new opportunities to explore the oceans in order to improve our understanding of the ocean habitat and climate change, develop early warning systems for earthquakes and tsunamis, monitor oil rigs, as well as to enhance the underwater warfare capabilities of the naval forces. Underwater Acoustic Sensor Networks (UASNs) are ideal for the harsh ocean environment where unmanned operation is desired for most of the missions. We proposed several localization algorithms and a mobility model for underwater sensor networks.
[WASA 2007] Melike Erol-Kantarci, L. Vieira, M. Gerla, `AUV-Aided Localization for Underwater Sensor Networks,` WASA special track on Underwater Sensor Networks, August 1-3, 2007, Chicago, IL.
[WUWNet 2007] M. Erol-Kantarci, L. Vieira, M. Gerla, `Localization with Dive`N`Rise (DNR) Beacons for Underwater Acoustic Sensor Networks,` The Second ACM International Workshop on UnderWater Networks WUWNet (in conjunction with ACM MobiCom 2007), September 14 2007, Montreal, Quebec, Canada.
[AHNJ 2011b] M. Erol-Kantarci, S. Oktug, L. Vieira, M. Gerla, `Performance Evaluation of Distributed Localization Techniques for Mobile Underwater Acoustic Sensor Networks`, Elsevier Ad Hoc Networks Journal, vol.9 no.1, 2011, pp. 61-72.
[INFOCOM 2008] A. Caruso, F. Paparella, L. Vieira, M. Erol-Kantarci, M. Gerla, `The Meandering Current Mobility Model and its Impact on Underwater Mobile Sensor Networks,` IEEE INFOCOM, 13-19 April 2008, Phoenix, AZ.
[COMMAG 2010] M. Erol-Kantarci, H. T. Mouftah, S. Oktug, `Localization Techniques for Underwater Acoustic Sensor Networks,` IEEE Communications Magazine, vol. 48, no.12, 2010.
[COMMST 2011] M. Erol-Kantarci, H. T. Mouftah, S. Oktug, `A Survey of Architectures and Localization Techniques for Underwater Acoustic Sensor Networks,` accepted for publication, IEEE Communications Surveys and Tutorials, 2011.
[AICT 2009] S. Kuruoglu, M. Erol-Kantarci, S. Oktug, `Localization in Wireless Sensor Networks with Range Measurement Errors,` The Fifth Advanced International Conference on Telecommunications (AICT), May 24-28, 2009, Venice, Italy. (Best Paper Award)
[GLOBECOM 2009] S. Kuruoglu, M. Erol-Kantarci, S. Oktug, `Three Dimensional Localization in Wireless Sensor Networks using the Adapted Multi-Lateration Technique Considering Range Measurement Errors`, Design and Development Forum in IEEE GLOBECOM, Hawaii, 2009.
Internet traffic is characterized by its long-range dependence and self-similarity which can lead to particularly large queuing delays and cause congestions to spread over a long time. The degree of self-similarity is measured by the Hurst parameter and several methods are available in the literature for estimating this parameter. In [TOIM 2011], we derived analytical expressions for widely used estimation methods in various domains for time series with periodic anomalies and demonstrated through simulations that periodicity-based anomalies negatively impact the Hurst parameter estimation methods.
[LNCS 2006] M. Erol-Kantarci, T. Akgul, S. Oktug, S. Baykut, `On the Use of Principle Component Analysis for the Hurst Parameter Estimation of Long-Range Dependent Network Traffic`, Lecture Notes on Computer Science, Vol.4263, pp. 464-473, 2006.
[TOIM 2011] T. Akgul, S. Baykut, M. Erol-Kantarci, S. Oktug, `Periodicity-based Anomalies in Self-similar Network Traffic Flow Measurements, ` IEEE Transactions on Instrumentation and Measurement, vol.60, no.4, pp.1358-1366, April 2011.